AD- A 142  741 


Research  Note  84-79 


EARLY  TRAINING  ESTIMATION  SYSTEM  (ETES) 
FINAL  REPORT 

APPENDIX  F:  USER’S  GUIDE 


Lawrence  H.  O'Brien,  Michael  Wagner  and  Beth  Modica 
Dynamics  Research  Corporation 


Charles  Jorgensen,  Contracting  Officer's  Representative 


Submitted  by 


§ 


Michael  H.  Strub,  Chief 
ARI  FIELD  UNIT  AT  FORT  BLISS,  TEXAS 

and 

Jerrold  M.  Levine,  Director 
SYSTEMS  RESEARCH  LABORATORY 


DTIC 

ELECTE 
JUL  9  1984 


\ 


3 


U.  S.  Army 

Research  Institute  for  the  Behavioral  and  Social  Sciences 

June  1984 


Approved  for  public  release;  distribution  unlimited. 


This  report,  it  submitted  by  the  connector,  hes  been  cleared  fcr  release  to  Defense  Technical  Information  Center 
(DTIC)  to  comply  with  regulatory  requirements.  It  hes  been  given  no  primery  distribution  other  then  to  OTIC 
end  will  be  eveileble  only  through  DTIC  or  other  reference  services  such  as  the  Nationel  Technical  Information 
Service  INTIS).  The  views,  opinion*,  and/or  findings  contained  in  this  report  ere  those  of  the  euthor(s)  end 
should  not  be  construed  as  an  official  Department  of  the  Army  position,  policy,  or  decision,  unless  so  designated 
by  other  official  documentation. 


84  06  rw 


THIS  DOCUMENT  IS  BEST 
QUALITY  AVAILABLE.  THE  COPY 
FURNISHED  TO  DTIC  CONTAINED 
A  SIGNIFICANT  NUMBER  OF 
PAGES  WHICH  DO  NOT 
REPRODUCE  LEGIBLY, 


REPORT  DOCUMENTATION  PAGE 


1.  REPORT. NUMBER 


Research  Note  84-79 


|*.  TlTLE-f*nd  Subtitle)  ' 

}  EARLY  TRAINING  ESTIMATION  SYSTEM:  FINAL  REPORT 

APPENDIX  F:  USER'S  GUIDE 


READ  INSTRUCTIONS 
BEFORE  COMPLETING  FORM 


1.  RECIPIENT'S  CATALOG  NUMBER 


5.  TYPE  OP  REPORT  A  PERIOD  COVERED 


FINAL 


6.  PERFORMING  ORG.  REPORT  NUMBER 


7.  AUTHOR!  A) 


S.  CONTRACT  OR  GRANT  NUMBER!*; 


Lawrence  H.  O'Brien,  Michael  Wagner,  Beth  Modica  I  MDA-903-80-C-0525 


».  PERFORMING  ORGANIZATION  NAME  AND  ADORESS 

DYNAMICS  RESEARCH  CORPORATION 

60  Concord  Street 

Wilmington,  Massachusetts  01887 


11.  CONTROLLING  OFFICE  NAME  ANO  ADORESS 

Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  Virginia  22333 


10.  PROGRAM  ELEMENT,  PROJECT.  TASK 
AREA  A  WORK  UNIT  NUMBERS 


2Q162722A791 


12.  REPORT  DATS 

June  1984 


11.  NUMBER  OF  PAGES 

412 


M.  MONITORING  AGENCY  NAME  A  ADDRESS (It  dHUrmnt  from  Controlling  Ollleo)  I  IS.  SECURITY  CLASS,  (al  this  rmport) 


US  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences 
Fort  Bliss  Field  Unit,  P.0.  Box  6057 
Fort  Bliss,  TX  7991 


16.  DISTRIBUTION  STATEMENT  (of  Olio  Report) 

Approved  for  public  release;  distribution  unlimited 


UNCLASSIFIED 


*?.  DISTRIBUTION  STATEMENT  (of  the  abstract  entered  in  Block  20,  It  different  from  Report ) 


U.  SUPPLEMENTARY  notes 


Charles  Jorgensen, Contracting  Officers  Representative 


IS.  KEY  WORDS  (Continue  on  reverse  etdm  If  necessary  and  Identity  by  block  number) 


Training  Training  Estimation 

Instructional  System  Development  Simulation 

Data  Base  Management  Training  Effectiveness 

Task  Analysis  Training  Effectiveness  Analysis 

_ _ Front  End  Analysis 


2X  ABSTRACT  (Coot£oum  so  revere*  aft tm  ft  neceeemry  mod.  Identify  by  block  number ) 


This  report  describes  the  research  and  development  activities  conducted  under 
the  Early  Training  Estimation  System  (ETES)  development  project.  The  Early 
Training  Estimation  System  (ETES)  is  an  integrated  set  of  procedures  and 
automated  tools  for  estimating  training  requirements  during  the  earliest 
phases  of  the  weapon  system  acquisition  process.  The  ETES  has  three  major 
components;  a  System  Description  Technology  (SDT) ,  Early  Training  Estimation 
Aids  and  Procedures  (TEAP) ,  and  Evaluative  Technology.  The  SDT  is  a  data 
base  management  system  for  storing  and  tracking  task  and  training-related 


PREFACE 


This  user’s  guide  is  part  of  the  Early  Training  Estimation 
System  (ETES) .  Development  of  the  ETES  was  sponsored  by  the 
Army  Research  Institute  (ARI)  under  contract  No.  MDA-903-80- 
C-0525.  Dynamics  Research  Corporation  ( DkC )  of  Wilmington, 
Massachusetts  was  the  contractor.  The  contract  monitor  for 
the  project  was  Dr.  Charles  Jorgensen.  The  guide  was 
written  by  Dr.  Lawrence  O'Brien,  Dr.  Michael  Wagner,  and  Ms. 
Beth  Modica. 


7 MU  *€■ 

j  by  ..  •  _  -  - 

!  i  '  ■:  '  *•  *  * 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


TABLE  OF  CONTENTS 


Section 

I.  Introduction  and  Summary 

1.1  Objectives 

1.2  Organization 

1.3  Need  for  Early  Training  Estimation 

1.4  Intended  Users  of  ETES 

1.5  ETES  Components 

1.5.1  The  SDT 

1.5.2  Training  Estimati  Aids/Procedures 
(TEAP) 

1.5.3  Evaluative  Technology 

1.6  Key  Concepts  Underlying  ETES 

1.6.1  Comparability  Analysis 

1.6.2  Phased  Versus  Steady-State  Training 
Resource  Requirements 

1.6.3  Limitations  of  Current  ETES 

1.7  Guidelines  for  Managing  ETES  Application 

1.7.1  ETES  Data  Management 

1.7.2  ETES  Study  Plan  Development 

1.8  Format  Used  to  Describe  ETES  Procedures 


1 


t 


TABLE  OF  CONTENTS  (continued) 

1 


I 


*4 


ft 


a 


» 


» 


Section  Page 

1.  Conduct  Functional  Requirements  Analysis  1-1 

1.1  Identify  Required  System  Functions  and 

Determine  MPT  Study  Requirements  1-2 

1.1.1  Identify  Acquisition  Characteristics  1-2 

1.1.2  Identify/Describe  System  Missions  1-6 

1.1.3  Determine  High-Level  Functions  1-11 

1.1.4  Document  Data  1-37 

1.2  Identify  System  Performance  Measures  and 

Goals  1-37 

1.2.1  Identify  System  Performance  Measures  1-41 

1.2.2  Identify  System  Performance  Goals  1-41 

1.2.3  Identify  Operational,  Organizational, 

and  Support  Concepts  1-45 

1.2.4  Document  Data  1-49 

1.3  Allocate  Required  System  Functions  and 

Identify  Lower  Level  Functions  1-49 

1.3.1  Allocate  Major  System  Functions/ 

Subfunctions  1-49 

1.3.2  Establish  Generic  Equipment  Structure  1-52 

1.3.3  Determine/Describe  Lower  Level 

Functions  1-57 


vi 


TABLE  OF  CONTENTS  (continued) 


Section  Page 


1.3.4 

Describe  Function  Sequence 

1-61 

1.3.5 

Document  Data 

1-64 

1.4 

Establish  The  Baseline  Comparison 

System 

(BCS) 

1-64 

1.4.1 

Establish  BCS 

1-67 

1.4.2 

Determine  BCS  Operational  and  Support 
Concepts 

1-71 

1.4.3 

Describe  Function  Sequence  and 

Information  Flow 

1-73 

1.4.4 

Document  Data 

1-73 

1.5 

Identify  Required  System  Improvements/ 

New  Technologies 

1-78 

1.5.1 

Identify  Required  System  Improvements 

1-78 

1.5.2 

Identify  New  Technologies 

1-80 

1.5.3 

Document  Data 

1-84 

1.6 

Format 

New  System 

1-84 

1.6.1 

Describe  Design  Differences 

1-85 

1.6.2 

Describe  New  System  Hardware/Software 

1-87 

1.6.3 

Document  Data 

1-87 

< 


1 


< 

] 


< 


1 


vi  i 


TABLE  OF  CONTENTS  (continued) 


I 

I 

I 

i 


Section  Page 

2.0  Generate  Tasks  2-1 

2.1  Identify,  Collect,  and  Format  Task  Data  2-1  * 

2.1.1  Identify  MOSs  Currently  Performing 

Similar  Functions  2-3 

2.1.2  Collect  and  Catalog  Candidate  MOS 

Task  Data  2-6 

2.2  Develop  Baseline  Comparison  System  (BCS) 

Task  List  2-6 

2.2.1  Select  Subsystem  from  BCS  Hardware/ 

Software  List  2-11 

2.2.2  Identify  Non-Army  Occupational 

Specialty  2-11 

2.2.3  Identify  MOSs  Associated  with 

Equipment  2-11 

2.2.4  Identify  Comparable  Army  Equipment  2-12 

2.2.5  Identify  Tasks  Associted  with 

Equipment  2-12 

2.2.6  Assess  Adequacy  of  Task  Descriptions  2-14 

2.2.7  Revise  Task  Descriptions  2-16 

2.2.8  Enter  Data  Into  Data  Base  2-16 


2.3  Assign  Baseline  Comparison  System  (BCS) 

Tasks  to  MOS,  Duty  Position,  and  Skill 

Level  2-16 

2.3.1  Identify  Duty  Position  Structure  2-19 


vi  i  i 


TABLE  OF  CONTENTS  (continued) 


Section  Page 

2.3.2  Assign  Tasks  to  Duty  Position  and 

Skill  Level  2-21 

2.3.3  Assign  Tasks  to  MOS  2-21 

2.3.4  Assess  Integrated  Impacts  of 

Assignments  2-22 

2.3.5  Document  Data  2-23 

2.4  Develop  New  System  Task  List  2-23 

2.5  Assign  New  System  Tasks  to  MOS,  Duty 

Position,  and  Skill  Level  2-35 

2.5.1  Identify  Duty  Position  Structure  2-38 

2.5.2  Assign  Tasks  to  Duty  Position  and 

Skill  Level  2-38 

2.5.3  Assign  Tasks  to  MOS  2-39 

2.5.4  Assess  Integrated  Impacts  of 

Assignments  2-40 

2.5.5  Document  Data  2-40 

3.0  Training  Requirements  Determination  3-1 

3.1  Select  Tasks  for  Training  3-3 

3.1.1  Choose  Task  Selection  and  Aggregation 

Methods  3-5 


ix 


TABLE  OF  CONTENTS  (continued) 


Section 

Page 

1 

3.1.2 

Rate/Score  Tasks  on  Relevant 

i 

Characteristics 

3-19 

3.1.3 

Apply  Algorithm/Aggregation  Method 

3-4  0 

3.1.4 

Review  Current  Task  Selection 

3-52 

4 

3.1.5 

Document  Data 

3-55 

( 

3.2 

Assign 

Tasks  to  Training  Settings 

3-55 

> 

3.2.1 

Select  Setting  Assignment  Method 

3-57 

4 

3.2.2 

Obtain  Task  Data 

3-58 

3.2.3 

Document  Data 

3-66 

i 

3.3 

Identify  Skills  and  Knowledges 

3-66 

< 

3.4 

Develop  Target  Population  Description 

3-72 

3.5 

Sequence  Tasks/Skills  For  Training 

3-76 

3.6 

Construct  Quasi-Program  of  Instruction 

3-80 

ri 

4 

3.6.1 

Identify  Courses  and  Course  Modules 

3-85 

3.6.2 

Identify  Methods  Within  Modules 

3-87 

3.6.3 

Identify  Module  Training  Times 

3-87 

► 

3.6.4 

Determine  Course  Length 

3-89 

I 

3.6.5 

Document  Data 

3-89 

3.7 

Assign 

Tasks  to  Media 

3-89 

> 

3.8 

Construct  Training  Paths 

3-91 

, 

► 


x 


I 


TABLE  OF  CONTENTS  (continued) 


i 


I 


r 


4 


» 


i 


Section 

4.  Estimate  Training  Resources 

4.1  Develop  Course  Operating  and  Support  Plan 

4.1.1  Determine  Course  Location 

4.1.2  Determine  Course  Frequency 

4.1.3  Determine  Usage  Rates 

4.1.4  Document  Data 

4.2  Determine  Number  of  Students  to  be  Trained 

4.3  Determine  the  Number  of  Instructors 

4.4  Determine  Facilities  Requirements 

4.5  Determine  Training  Device  and  Training 
Equipment  Requirements 

4.6  Determine  Requirements  for  Other  Training 
Resources 

4.6.1  Determine  Number  of  Training  Aids/ 
Instructional 

4.6.2  Determine  Ammunition  Requirements 
5.  Determine  Training  Cost 

5.1  Estimate  Individual  Course  Cost 

5.2  Aggregate  Course  Costs 


Page 

4-1 

4-3 

4-3 

4-7 

4-7 

4-10 

4-10 

4-20 

4-20 

4-26 

4-32 


4-32 

4- 38 

5- 1 

5-1 

5-5 


xi 


TABLE  OF  CONTENTS  (continued) 


Section  Page 

6.  Estimate  Training  Efficiency/Effectiveness  6-1 

6.1  Estimate  Training  Efficiency  6-1 

6.1.1  Estimate  Training  Efficiency  6-1 

6.1.2  Estimate  Media  Efficiency  6-4 

6.1.3  Estimate  Training  Method  Efficiency  6-6 

6.2  Estimate  Training  Effectiveness  6-28 

7.  Evaluate  Training  Program  7-1 

7.1  Develop  Figures-of-Merit  7-1 

7.1.1  Assess  Congruence  with  Training 

Development  Guidelines  7-7 

7.1.2  Assess  Congruence  with  Program 

Requirements  7-11 

7.1.3  Assess  Training  Complexity  7-16 

7.1.4  Assess  Feasibility  7-18 

7.1.5  Construct  Summary  Evaluation  Score  7-19 

7.2  Identify  Likely  Problem  Areas  7-25 

7.3  Identify  Likely  Causes  of  Problem  Areas  7-30 

7.4  Identify  Alternatives  7-37 

7.5  Evaluate  Alternatives  7-45 

7.6  Evaluate  Impact  of  System  Changes  7-52 


x  1  1 


TABLE  OF  CONTENTS  (continued) 


Section  Page 

8.  Develop  Inputs  to  Acquisition  Process  8-1 

8.1  Develop/Monitor  Training  Development  Schedule  8-1 

8.2  Develop  Impacts  to  Acquisition  Processes 

and  Documents  8-1 

8.2.1  Develop  Inputs  to  OICTP/ICTP  8-2 

8.2.2  Develop  Inputs  to  Cost  and  Training 

Effectiveness  Analyses  8-6 

8.2.3  Develop  Inputs  to  Logistics  Support 

Analysis  8-14 

8.2.4  Develop  Inputs  to  Operational  Testing  8-14 

8.2.5  Develop  Inputs  to  Training  Device 
Requirements  Documents  and  Processes  8-19 

8.2.6  Develop  Inputs  to  New  Equipment 

Training  8-20 

8.2.7  Develop  Inputs  to  System  Requirements 

Documents  8-21 

8.2.8  Develop  Inputs  to  Request  for  Proposal 

Development/Evaluation  8-21 

8.2.9  Develop  Inputs  to  Personnel  Documents/ 

Processes  8-23 

8.2.10  Develop  Inputs  to  System  Level 

Documents  and  Processes  8-24 

8.2.11  Develop  Inputs  to  Mission  Area 

Analysis/Soldier  Analysis  8-28 

8.2.12  Relationship  Between  ETES  and  LCSMM  8-28 

xi  i  i 


TABLE  OF  CONTENTS 


1 

Section 

Page 

a 

References 

rl 

Appendix 

A: 

Overview  of  Media  Selection  Program 

A-l 

Appendix 

B: 

Overview  of  Automated  Resource  and  Cost 

Estimation  Technique 

B-l 

Appendix 

C: 

Overview  of  Automated  Planning  and 

• 

Scheduling  Technique 

C-l 

Appendix 

D: 

Description  of  Projects  Related  to  ETES 

D-l 

Appe ndix 

E: 

Data  Source  Index 

E-l 

Appendix 

F: 

List  of  Performance  Measures 

F-l 

1 

Appendix 

G: 

Description  of  TRAINVICE 

G-l 

Appendix 

H: 

Training  Concept  Descriptions 

H-l 

Appendix 

Is 

HRTES  Procedures  for  Identifying  OT 

Test  Issues 

1-1 

m 


xiv 


LIST  OF  FIGURES 


Figure  Page 

1-1  System  Definition  as  a  Function  of  Acquisition 

Process  1-5 

1-2  ETES  Components  1-7 

1-3  ETES  Architecture  1-13 

1-4  System  Development  Process  for  SDT  1-15 

1-5  ETES  Training  Estimation  Aids/Procedures  1-19 

1-6  Relationship  Between  ISD  and  ETES  1-29 

1-7  ETES  Evaluative  Technology  1-30 

1-8  Overview  of  Comparability  Analysis  Procedures  1-35 

1-9  ETES  Data  Management.  1-43 

I — 1 0  Overview  of  Audit  Trail  Procedures.  1-48 

1-11  Constructing  a  Study  Plan.  1-58 

1-12  Procedures  for  Developing  Data  Source  Index  1-60 


1-1  Overview  Diagram:  Identify  Required  System 

Functions  and  Determine  MPT  Study  Requirements 
(1.1)  1-3 

1-2  Acquisition  Information  Worksheet  (ACQ)  1-7 

1-3  Mission  Profile  Worksheet  (MIS)  1-9 

1-4  System  Performance  Specification  Worksheet  (PRF)  1-12 

1-5  Overview  Diagram:  Identify  System  Performance 

Measures  and  Goals  (1.2)  1-40 

1-6  System  Performance  Specification  Worksheet  (PRF)  1-42 

1-7  Organizational,  Operational,  and  Support  Concepts 

Worksheet  (OSC)  1-46 

1-8  Operational  Environment  Worksheet  (ENV)  1-48 


xv 


LIST  OF  FIGURES  (continued) 


► 

M 


B 

Figure 

Page 

1-9 

Overview  Diagram:  Allocate  Required  System 
Functions  and  Identify  Lower  Level  Functions 

(1.3) 

1-51 

*i 

1-10 

System  Performance  Specif ication- Worksheet  (PRF) 

1-53 

1-11 

Generic  System  Worksheet  (GEN) 

1-58 

1-12 

Function  Sequence  Worksheet 

1-62 

1-13 

Example  Function  Sequence  Diagram 

1-63 

• 

1-14 

Overview  Diagram:  Establish  the  Baseline 
Comparison  System  (BCS)  (1.4) 

1-66 

a 

1-15 

Baseline  Comparison  Equipment  Selection 

Worksheet  (EQP) 

1-68 

1-16 

BCS  Equipment  Description  Worksheet 

1-72 

1-17 

Function  Sequence  Worksheet  (FSQ) 

1-75 

a 

1-18 

Overview  Diagram:  Identify  Required  System 
Improvements/New  Technologies  (1.5) 

1-79 

1-19 

Baseline  Comparison  Performance  Shortfall 

Worksheet  (PSH) 

1-81 

1-20 

Overview  Diagram:  Format  New  System  (1.6) 

1-86 

1-21 

Design  Difference  Worksheet 

1-88 

• 

1-22 

New  System  Equipment  Description  Worksheet  (EQP) 

1-89 

2-1 

Overview  Diagram:  Generate  Tasks  (2.0) 

2-2 

2-2 

Identify,  Collect,  and  Format  Task  Data 

2-4 

• 

2-3 

Candidate  MOS  Worksheet  (CND) 

2-5 

2-4 

Overview  Diagram:  Develop  BCS  Task  List  (2.2) 

2-8 

2-5 

BCS  Task  Association  Worksheet  (BCST) 

2-10 

xvi 


LIST  OF  FIGURES 


Figure  Page 

2-6  Overview  Diagram:  Assign  BCS  Tasks  to  MOS, 

Duty  Position,  and  Skill  Level  (2.3)  2-20 

2-7  Overview  Diagram:  Develop  New  System  Task 

List  (2.4)  2-25 

2-8  New  System  Task  Generation  Worksheet  ( NTA )  2-27 

2-9  Overview  of  Relationship  Between  Subsystem 

Functions,  Collective  Tasks,  and  Individual 
Tasks  2-30 

2- 10  Overview  Diagram:  Assign  BCS  Tasks  to  MOS, 

Duty  Position,  and  Skill  Level  (2.5)  2-37 

3- 1  Overview  Diagram:  Estimate  Training  3-2 

3-2  Overview  Diagram:  Select  Tasks  for  Training  3-4 

3-3  Choose  Task  Selection  and  Aggregation  Methods 

(3.1.1) .  3-7 

3-4  DIF  Model  3-11 

3-5  Detailed  DIF  3-12 

3-6  Rate/Score  Tasks  on  Relevant  Characteristics 

(3.1.2)  3-20 

3-7  DIF  Data  Collection  Format  3-21 

3-8  Apply  Algorithm/Aggregation  Method  (3.1.3)  3-46 

3-9  Review  Current  Task  Selection  (3.1.4)  3-54 

3-10  Overview  Diagram:  Assign  Tasks  to  Training 

Settings  (3.2)  3-56 

3-11  Overview  Diagram:  Make  Setting  Assignments 

(3. 2. 2. 3)  3-63 


xv  i  i 


LIST  OF  FIGURES  (continued) 


Figure 

3-12  Overview  Diagram:  Identify  Skills  and 
Knowledges  (3.3) 

3-13  Overview  Diagram:  Sequence  Tasks/Skills  for 
Training 

3-14  Overview  Diagram:  Construct  QUASI-POI  (3.6) 

3-15  Overview  Diagram:  Construct  Training  Paths 

3- 16  Example  of  Training  Path  for  Institutional 

Courses 

4- 1  Overview  Diagram:  Determine  Course  Operating 

and  Support  Plan 

4-2  Operating  and  Support  Plan  Worksheet  (OPSP) 

4-3  Media  Usage  Rate  Worksheet (MUS ) 

4- 4  Overview  Diagram:  Determine  Course  Locations 

5- 1  Overview  Diagram:  Determine  Training  Cost 

(5.1) 

5-2  Individual  Course  Cost  Worksheet  (ICOST) 

5- 3  Course  Aggregation  Worksheet  (ACOST) 

6- 1  Overview  Diagram:  Estimate  Training  Efficiency/ 

Effectiveness 

6-2  Overview  Diagram:  Estimate  Training  Efficiency 

6-3  Overview  Diagram:  Determine  Training  Efficiency 

6-4  Overview  Diagram:  Trainvice  Procedures 

6-5  Overview  Diagram:  Generalized  Procedures  for 
Assessing  Efficiency  of  Training  Methods 


xvi  i  i 


LIST  OF  FIGURES  (continued) 


Figure 


6- 6  Overview  Diagram:  Estimate  Training 

Effectiveness 

7- 1  Overview  Diagram:  Evaluate  Training  Program 

(7.0) 

7-2  Overview  of  Relationships  Among  Training 
Figures  of  Merit 

7-3  Overview  Diagram:  Construct  Measure  of 
Congruence  between  Actual  and  Prescribed 
Training  Development  Practices 
7-4  Overview  Diagram:  Assess  Feasibilty  of 
Training  Program  Events 

7-5  Overview  Diagram:  Identify  Problem  Areas  (7.2) 
7-6  Relationship  between  Two  Types  of  Causes 
of  Training  Problems 

7-7  Overview  Diagram:  Identify  Likely  Causes  (7.3) 
7-8  Overview  of  Causal  Structure  Underlying 
Training  Cost 

7-9  Overview  of  Causal  Structure  Underlying 
Training  Complexity  (Training  Time) 

7-10  Overview  of  Causal  Structure  Underlying 
Early  Measure  of  Effectiveness 
7-11  Overview  Diagram:  Identify  Alternatives  (7.4) 
7-12  Overview  of  Procedure  for  Evaluating  Training 
Alternatives 

7-13  Overview  Diagram:  Conduct  Sensitivity  Analysis 


Page 

6- 30 

7- 2 

7-6 

7-8 

7-20 

7-27 

7-31 

7-33 

7-34 

7-35 

7-36 

7-39 

7-46 

7-48 


xix 


LIST  OF  FIGURES  (continued) 

Figure 

8-1  Basic  HRTES  Flow 

8-2  LCSMM  Events 


> 

■ 

i 

LIST  OF  TABLES 

< 

1 

Table 

Page 

1-1 

SDT  Data  Elements 

1-10 

1-2 

Major  Army  Acquisition  Processes  and  Documents 
Related  to  Early  Training  Estimation 

1-33 

4 

*  1-3 

Sample  Page  From  Data  Source  Index 

1-45 

< 

1-4 

Example  Page  from  Library  Index 

1-46 

1-5 

Example  of  ETES  Worksheet  -  System  Performance 
Specification  Worksheet  (PRF) 

1-50 

» 

1-6 

List  of  ETES  Worksheets 

1-51 

< 

1-7 

ETES  Study  Plan  Outline  (OUT) 

1-53 

1-8 

Logical  Sequence  for  Applying  ETES  Functions/ 

Procedures 

1-59 

f 

1-9 

Organization  of  Remaining  Sections 

1-63 

< 

1-1 

Outline  of  Information  Elements  Contained  in 

JMSNS 

1-4 

1 

1-2 

List  of  Program  Documents  for  a  New  System 

1-5 

1-3 

Systems  and  Functional  Area  Proponents 

1-16 

1-4 

Doctrinal  Literature  Program  Publications  and 
Proponents 

1-27 

• 

1-5 

Guidelines  for  Entering  Data  Into  SDT  (1.1) 

1-38 

4 

1-6 

Definition  of  Key  Concepts 

1-39 

1-7 

Guidelines  for  Entering  Data  Into  SDT  (1.2) 

1-50 

1-8 

Example  of  Generic  Equipment  Structure 

1-56 

• 

1-9 

Guidelines  for  Entering  Data  Into  SDT  (1.3) 

1-65 

1 

1-10 

Example  of  an  Equipment  Taxonomy 

1-70 

• 

1-11 

Format  of  the  Operational  Concept  and 

Concept  Statement 

1-74 

( 

• 

k.  . 

xxi 

4 

LIST  OF  TABLES  (continued) 


Table  Page 

1-12  Guidelines  for  Entering  Data  Into  SDT  (1.4)  1-76 

1- 13  Guidelines  for  Entering  Data  Into  SDT  (1.6)  1-90 

2- 1  LCSMM  Documents  and  Processes  Related  to  Early 

Training  Estimation  2-7 

2-2  Relationship  Between  OICTP  and  ETES  Functions  2-13 

2-3  Relationship  Between  CTEA  and  ETES  Functions  2-15 

2-4  Summary  of  Standardized  Reports  Related  to 

Early  Training  Estimation  2-17 

2-5  Organizations  Interviewed  During  Task  1  2-17 

2-6  Potential  Problem  Solving  Aids  2-24 

2-7  Task  Deletion/Modification/Addition  Codes  2-29 

2-8  Maintenance  Actions  2-31 

2-9  Example  Maintenance  Allocation  Chart  2-34 

2-10  Guidelines  for  Entering  Data  Into  SDT  (2.4)  2-36 

2- 11  Guidelines  for  Entering  Data  Into  SDT  (2.5)  2-41 

3- 1  Overview  of  Task  Selection  Methods  3-6 

3-2  Data  Sources  for  4-Factor  Model  3-14 

3-3  Potential  Algorithms  for  Aggregating  Task 

Factors  3-17 

3-4  Appropriate  Aggregation  Algorithms  for 

Alternative  Task  Selection  Models  3-18 

3-5  Sample  Data  Set  Using  4-Factor  Model  3-28 

3-6  Sample  Data  Collection  Forms  for  8-Factor 

Model  3-29 


LIST  OF  TABLES  (continued) 


Table 

3-7 

3-8 

3-9 

3-10 

3-11 

3-12 

3-13 

3-14 

3-15 

3-16 

3-17 

3-18 

3-19 

3-20 

3-21 

3-22 

3-23 

3-24 

3- 25 

4- 1 

4-2 


Page 


Sample  Data  Collection  Forms  for  8-Factor 
Model  3-41 

Data  Summary  Worksheets  3-48 

Worksheet  for  Selecting  Tasks  for  Training 
(TSEL)  3-53 

Data  Summary  Worksheet  for  Training  Setting 
Assignment  (SUMSET)  3-61 

Aggregation  Methods  for  Assigning  Tasks  to 
Training  Settings  3-65 

Guidelines  for  Entering  Data  into  SDT  (3.2)  3-67 

Skill  and  Knowledge  Worksheet  (SKILL)  3-69 

Skill  &  Knowledge  Data  Sources  3-71 

Guidelines  for  Entering  Data  into  SDT  (3.3)  3-74 

Procedures  for  Developing  Target  Population 
Description  Elements  3-75 

Target  Population  Worksheet  (POP)  3-77 

Task/Skill  Sequence  Worksheet  (SEQ)  3-81 

QPOI  Worksheet  -  PARTS  1  &  2  (QPOI-1,  QPOI-2)  3-82 

Course  Modification  Worksheet  (MOD)  3-86 

Instructional  Methods  3-88 

Guidelines  for  Entering  Data  into  SDT  (3.6)  3-90 

Guidelines  for  Entering  Data  into  SDT  (3.7)  3-92 

Training  Path  Worksheets  (PATH)  3-95 

Guidelines  for  Entering  Data  into  SDT  (3.8)  3-96 


Algorithm  for  Determining  Number  of  Course 
Starts  and  Sessions  Per  Site 

Methods  of  Instruction  and  Associated  Student/ 
Instructor  Ratios 


4-8 


x  xi  i  i 


4 


« 


1 

1 


4-9 


LIST  OF  TABLES  (continued) 


Table  Page 

I  4-3  Guidelines  for  Entering  Data  into  SDT  (4.1)  4-11 

4-4  Algorithm  for  Determining  Steady-State  Training 

Input  Requirements  4-12 

4-5  Number  of  Students  -  Steady-State  Worksheet 
I  (NSSS)  4-13 

4-6  Worksheet  for  Phased  Course  Input  Requirements  4-14 

4-7  Number  of  Students  to  be  Trained  Worksheet 

(NSTUD)  4-21 

4-8  Guidelines  for  Entering  Data  into  SDT  (4.2)  4-22 

4-9  Algorithm  for  Determining  Number  of  Instructors  4-23 

4-10  Number  of  Instructors  Worksheet  (NINS)  4-24 

1  4-11  Guidelines  for  Entering  Data  into  SDT  (4.3)  4-25 

4-12  Algorithm  for  Determining  Training  Facilities 

Requirements  4-27 

4-13  Worksheet  for  Determining  Training  Facilities 
I  ( DFAC )  4-28 

4-14  Facilities  Requirements  Worksheet  (NFAC)  4-29 

4-15  Algorithm  for  Determining  Number  of  Training 

Devices/  Equipments  Per  Site  4-30 

4-16  Worksheet  for  Determining  Number  of  Training 

Devices/Equipments  (TDEV)  4-31 

4-17  Training  Device/Equipment  Worksheet  (DEEQ)  4-33 

4-18  Guidelines  for  Entering  Data  into  SDT  (4.5)  4-34 

4-19  Worksheet  for  Determining  Number  of  Media  and 

Instructional  Aids  (NMED)  4-36 

4-20  Number  of  Media/Aids  (MEDSUM)  4-37 

4-21  Guidelines  for  Entering  Data  into  SDT  (4.6)  4-39 

4-22  Ammunition  Requirements  Worksheet  (AMREQ)  4-40 

4-23  Ammunition  Requirements  Summary  Sheet  (AMSUM)  4-41 


< 


4 


* 


xx  IV 


LIST  OF  TABLES  (continued) 


Table 

5- 1 

6- 1 
6-2 
6-3 
6-4 
6-5 
6-6 

6-7 

6-8 

6-9 

6-10 


7-1 

7-2 

7-3 


7-4 

7-5 

7-6 


Page 


Guidelines  for  Entering  Data  into  SDT  (5.1)  5-6 

Training  Summary  Worksheet  (TVICE)  6-9 

Guidelines  for  Entering  Data  into  SDT  (6.1.2)  6-10 

Instructional  Methods  6-13 

Methods  Related  Variables  6-14 

Method  Rating  Worksheet  (METH)  6-25 

Task  Rating  Worksheet  for  Instructional  Methods 
(TAMOD)  6-26 

Worksheet  for  Calculating  Method  Efficiency 
(METHEFF)  6-27 

Form  for  Rating  Effectiveness  6-31 

Guidelines  for  Developing  Elements  in  Training 
Effectiveness  6-33 

Worksheet  for  Calculating  Overall  Effectiveness 
Score  (EFF)  6-34 

Overview  Diagram:  Evaluate  Training  Program 

(7.0)  7-3 

Overview  of  Relationships  Among  Training 

Figures  of  Merit  7-5 

Overview  Diagram:  Construct  Measure  of 

Congruence  between  Actual  and  Prescribed 

Training  Development  Practices  7-10 

Overview  Diagram:  Assess  Feasibilty  of 

Training  Program  Events  7-12 

Overview  Diagram:  Identify  Problem  Areas  (7.2)  7-14 

Relationship  between  Two  Types  of  Causes  of 
Training  Problems  7-15 


i 


l 


4 


4 


Xxv 


4 


LIST  OF  TABLES  (continued) 


Table 

7-7 

7-8 

7-9 

7-i  0 

7-11 

7-12 

7-13 
7-14 
7-1  5 
7-16 
7-17 
7-18 
7-19 

7-20 


8-1 

8-2 

8-3 

8-4 

8-5 

8-6 


Page 


Overview  Diagram:  '  Identify  Likely  Causes  (7.3)  7-17 

Overview  of  Casual  Structure  Underlying 

Training  Cost  1-21 

Overview  of  Casual  Structure  Underlying 

Training  Complexity  (Training  Time)  7-22 

Overview  of  Causal  Structure  Underlying 

Early  Measure  of  Effectiveness  7-24 

Overview  Diagram:  Identify  Alternatives  (7.4)  7-28 

Overview  of  Procedure  for  Evaluating  Training 

Alternatives  7-29 

Overview  Diagram:  Conduct  Sensitivity  Analysis  7-38 

Training  Section  Alternatives  7-40 

Training  Method  Alternatives  7-42 

Training  Media  Alternatives  7-43 

Training  Project  Elements  7-44 

Listing  of  Candidate  Parameters  7-50 

Worksheet  tor  [Describing  Results 

of  Sensitivity  Analysis  (ETES)  7-53 

ETES  Procedures  Which  Must  be  Reapplied  In 

Response  to  System  Changes  7-54 

LCSMM  Documents  and  Processes  Related  to  Early 
Training  Estimation  8-3 

Relationship  Between  OICTP  and  ETES  Functions  8-7 

Relationship  Between  Institutional  Training 
Concept  Elements  and  ETES  Functions  8-9 

Relationship  Between  CTEA  and  ETES  Functions  8-12 

LSAR  Elements  Included  in  ETES  SDT  8-15 

Relationship  Between  HRTES  Function  and  ETES 
Function  8-18 


XXV  1 


LIST  OF  TABLES  (continued) 


Table 

8-7  ETES-LOA  Relationships 

8-8  ETES-QQPRI  Relationships 

8-9  ETES-IPS  Relationships 

8-10  ETES-Soldier  Analysis  Relationships 


Page 


8-22 

8-25 

8-26 

8-29 


xxv  1 1 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


xv  i  i  i 


SECTION  I 


INTRODUCTION  AND  SUMMARY 


1.1  OBJECTIVES 

This  guide  describes  the  manual  procedures  in  the  Early 
Training  Estimation  System  (ETES).  The  objective  of  ETES  is 
to  provide  an  integrated  set  of  procedures  and  automated 
tools  for  estimating  training  requirements  for  emerging  Army 

weapons  systems  during  the  earliest  phases  of  the 

acquisition  process  (Mission  Area  Analysis,  Concept 
Exploration,  and  Demonstration  and  Validation). 

Detailed  descriptions  of  ETES  automated  tools  are  available 
in  the  following  documents :1 

o  User's  Guide:  ETES  System  Description  Technology 
(approximately  300  pages) 

o  User's  Guide;  ETES  Media  Selection  Program 

(approximately  200  pages) 

o  User's  Guide;  ETES  Resource  and  Cost  Estimation 

Technique  (approximately  100  pages) 

°  User's  Guide:  ETES  Automated  Training,  Planning 

and  Scheduling  Technique  (approximately  100  pages) 


x  For  information  on  obtaining  ETES  documents,  contact  Dr. 
Lawrence  O'Brien,  Dynamics  Research  Corporation,  7  Lopez 
Road,  Wilmington,  MA  01887. 


A  description  of  the  research  which  led  to  the  development 
of  ETES  is  provided  in  the  ETES  Final  Report.  Additional 
descriptions  of  ETES  tools  are  provided  in  the  following 
technical  papers: 

Jorgensen,  C.  ,  and  O'Brien,  L.  The  Early  Training  Estimation 
System:  An  automated  training  needs  assessment  technique. 
(Paper  to  be  published  in  the  first  issue  of  the  Training 
Technology  Journal ) . 

O'Brien,  L.  Automated  System  Description  Technology.  Paper 
presented  at  the  Second  Annual  Conference  on  Microcomputers 
in  Education,  Washington,  June  1982. 

Boylston,  D.  An  automated  decision  aid  for  the  assignment 
of  tasks  to  training  media  in  early  training  estimation. 
Paper  to  be  presented  at  the  Slst  Symposium  of  the  Military 
Operations  Research  Society,  September  27-29,  1983. 

O'Brien,  L.  Early  Training  Estimation  System  (ETES). 
Proceedings  of  the  TRADOC  Developments  Institute  Chiefs  of 
Analysis  Seminar,  October  21-13,  1981. 

1.2  ORGANIZATION 

The  remainder  of  this  section  describes  (a)  the  requirements 
which  led  to  the  development  of  ETES,  (b)  the  intended  users 
of  ETES,  (c)  the  components  of  ETES,  (d)  key  concepts 
underlying  the  ETES  procedures,  and  (e)  guidelines  for 
managing  an  ETES  application. 


Sections  1  thru  6  describe  the  ETES  Training  Estimation  Aids 
and  Procedures.  Sections  7  and  8  describe  the  procedures  in 
the  ETES  Evaluative  Technology. 


Overviews  of  the  three  ETES  Automated  Training  Estimation 
Tools  —  the  Media  Selection  Program,  the  Resource  and  Cost 
Estimation  Technique  ( RCET) ,  and  the  Automated  Planning  and 
Scheduling  Technique  are  provided  in  Appendices  A,  B,  and  C, 
respectively.  An  overview  of  other  Army  Research  Institute 
projects  related  to  ETES  is  provided  in  Appendix  D.  A  data 
source  index  describing  the  most  common  data  sources  which 
are  used  in  an  ETES  application  is  provided  in  Appendix  E. 

1.3  NEED  FOR  EARLY  TRAINING  ESTIMATION 

The  Early  Training  Estimation  System  provides  a  capability 
for  systematically  estimating  training  requirements  for 
developing  Army  weapon  systems  during  the  earliest  phases  of 
the  acquisition  process  (Mission  Area  Analysts,  Concept 
Exploration  -  Phase  I,  and  Demonstration  and  Validation  - 
Phase  II).  These  estimates  of  training  requirements  include 
specification  of  the  system's  task  requirements,  training 
course  requirements,  resource  requirements,  and  estimates  of 
training  cost  and  "effectiveness." 

There  are  two  major  reasons  why  early  estimates  of  training 
requirements  are  needed.  First,  by  developing  earlier  and 
more  accurate  estimates  of  training  requirements,  the 
training  planning  process  can  begin  earlier.  As  a  result, 
the  training  products  associated  with  a  system,  many  of 
which  require  a  long  lead  time,  are  more  likely  to  be 
available  when  the  system  is  fielded. 


z  ETES  only  provides  gross  high  level  estimates  of 
training  effectiveness.  These  estimates  are  only 

appropriate  during  the  earliest  phases  of  the  acquisition 
process. 


Second,  by  developing  estimates  of  training  requirements  for 
the  various  design  alternatives  which  are  considered  during 
early  phases  of  the  acquisition  process,  the  training 
developer  can  provide  the  information  needed  to  effectively 
influence  system  design. 

The  importance  of  obtaining  training  projections  during  the 
earliest  phases  of  system  acquisition  cannot  be  over¬ 
estimated.  Most  of  the  major  design  decisions  related  to  a 
new  system  are  made  during  the  early  phases  of  the 
acquisition  process  (see  Figure  1-1).  Thus,  if  training  is 
to  influence  design,  it  must  impact  these  early  design 
decisions.  And  there  is  good  reason  for  ensuring  that 
training-related  considerations  do,  in  fact,  impact 
design . 

In  most  weapon  systems,  manpower  costs,  including  training 
costs,  are  the  largest  component  of  the  system's  operation 
and  support  costs.  Because  these  costs  are  the  result  of 
demands  generated  by  the  design  characteristics  of  a  system, 
acquisition  policies  have  been  established  by  the  Federal 
Government  to  ensure  that  support  requirements  are 
accurately  determined  and  evaluated  in  conjunction  with 
system  development  (for  example,  the  DoDD  5000  series  on 
major  systems  acquisition).  ETES  is  specifically  designed  to 
provide  the  Army  with  the  capability  for  meeting  the 
training-related  requirements  specified  in  these  acquisition 
policies. 

1.4  INTENDED  USER  OF  ETES 

As  part  of  the  review  of  existing  Army  procedures  conducted 
during  Task  1  of  the  ETES  development  study,  potential  users 


95% 


Figure  1-1.  System  Definition  as  a  Function  of  Acquisition  Phase. 


for  an  early  training  estimation  system  were  identified. 
The  primary  user  organizations  of  ETES  are  expected  to  be 
(1)  the  training  developers  in  the  Army  schools  associated 
with  the  development  of  new  systems,  (2)  program  management 
offices  (PMOs)  for  new  systems,  particularly  those  indi¬ 
viduals  concerned  with  training  development  or  Integrated 
Logistics  Support,  (3)  the  TRADOC  System  Manager  (TSM) ,  (4) 
other  Army  organizations  concerned  with  training  development 
such  as  the  TRADOC  System  Analysis  Activity  (TRASANA)  and  PM 
TRADE,  and  (5)  contractors  who  must  develop  training 
requirements  for  new  systems. 

1.5  ETES  COMPONENTS 

ETES  has  three  major  components;  a  System  Description 
Technology  ( SDT) ,  Early  Training  Estimation  Aids  and 
Procedures  (TEAP) ,  and  Evaluative  Technology  (see  Figure  I- 
2).  The  SDT  is  a  data  base  management  system  for  storing 
and  tracking  task  and  training-related  data.  The  data  in 
the  SDT  is  used  in  the  TEAP  to  estimate  training 
requirements  for  a  new  system,-^  These  training  requirements 
include  estimates  of  task  requirements,  course  requirements, 
and  resource  requirements  as  well  as  estimates  of  training 
costs,  training  efficiency,  and  training  "effectiveness". 
In  the  Evaluative  Technology  component,  the  integrated 
impacts  of  training  requirements  are  assessed,  training 
alternatives  are  evaluated,  tradeoffs  and  sensitivity 
analyses  of  key  parameters  are  conducted,  and  the 
relationships  between  ETES  outputs  and  key  Army  acquisition 
documents  and  processes  are  specified. 

More  details  on  the  three  components  of  ETES  are  provided  in 
the  sections  which  follow.  A  detailed  description  of  the 
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ETES  Components. 


SDT  is  provided  in  User's  Guide:  ETES  SDT.  Detailed 
descriptions  of  the  Training  Estimation  Aids  and  Procedures 
are  provided  in  Sections  1  thru  6  of  this  guide.  Detailed 
descriptions  of  the  Evaluative  Technology  are  provided  in 
Sections  7  and  8  of  this  guide. 

There  are  three  automated  aids  associated  with  the  Training 
Estimation  Aids/Procedures  and  Evaluative  Technology;  (1) 
the  Media  Selection  Program  which  is  an  automated  tool  for 
assigning  tasks  to  media,  (2)  the  Resource  and  Cost 
Estimation  Technique  which  is  an  automated  tool  for 
estimating  instructor  requirements  and  course  costs,  and  (3) 
the  Automated  Planning  and  Scheduling  Technique  (APST)  which 
is  an  automated  tool  for  describing  and  monitoring  the 
training  development  schedule.  Detailed  descriptions  of 
these  three  automated  tools  are  provided  in  User's  Guide: 
Media  Selection  Program;  User's  Guide:  Resource  and  Cost 
Estimation  Technique;  and  User's  Guide;  Automated  Planning 
and  Scheduling  Technique. 

1.5.1  The  SDT 


The  SDT  is  a  microcomputer-based  data  base  management  system 
for  (1)  describing  actual  and  projected  system  elements 
including  system  functional  requirements,  system  hardware 
and  software  concepts,  tasks,  skills,  and  training  program 
elements  and  their  associated  resource  and  cost  require¬ 
ments,  (2)  for  storing  the  above  information,  (3)  for 
changing  and  updating  this  information,  and  (4)  for  trans¬ 
mitting  the  information  among  the  participants  in  the 
acquisition  process.  The  DT  data  base  management  system  is 
designed  to  allow  training  developers  to  keep  track  of  the 
numerous  system  changes  which  occur  during  the  early  phases 


Table  1-1  SDT  Data  Elements 
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of  the  acquisition  process.  In  addition,  it  provides  a 
centralized  data  base,  thus  eliminating  the  redundant  data 
collection  efforts  which  typically  take  place  among  the 
numerous  training  development  organizations  within  the  Army. 

The  computerized  SDT  also  facilitates  the  use  of  automated 
aids  for  estimating  training  program  elements  and  their 
associated  resource  and  cost  requirements.  These  automated 
aids  allow  the  training  developer  to  quickly  develop 
training  requirements  estimates  for  alternative  training  and 
system  concepts.  This  capability  greatly  increases  the 
probability  that  training  requirements  will  be  effectively 
considered  during  the  early  phases  of  the  weapon  system 
development  process. 

I. 5. 1.1  SDT  Data  Elements 


To  provide  an  effective  communication  vehicle  for  training 
developers  and  other  participants  in  the  acquisition 
process,  the  SDT  has  been  designed  to  describe  (a)  training 
programs  and  their  associated  resources,  (b)  the  tasks  which 
drive  these  training  programs,  (c)  the  personnel  who  will  be 
required  to  perform  the  tasks,  (d)  the  system  designs  which 
generate  the  task  requirements,  and  (e)  the  functional 
requirements  for  which  the  system  designs  have  been 
developed.  An  overview  of  the  major  data  elements  currently 
included  in  the  SDT  is  provided  in  Table  1-1.  Although  the 
data  elements  in  the  SDT  are  designed  to  remain  fixed  for 
any  particular  application,  the  SDT  software  allows  the  SDT 
data  base  manager  to  quickly  change  the  data  elements  to 
meet  the  needs  of  particular  users  and/or  models. 


1.5. 1.2  Characteristics  of  SDT  Data  Base 


Data  base  management  systems  such  as  the  SDT  use  a 
specialized  ’’language"  to  describe  the  relationships  among 
data  elements.  Unlike  many  other  data  base  management 
systems,  the  SDT  does  not  require  the  user  to  explicitly  use 
or  even  know  about  this  specialized  data  language.  In  the 
SDT,  the  data  language  is  made  "invisible"  to  the  user 

through  "user-friendly"  human-computer  dialogue  techniques 
such  as  menu  selection  and  question-and  answering. 

There  are  four  major  types  of  variables  in  the  SDT  data 

language : 

o  Entities  -  Major  system  elements.  Entities  are 
roughly  equivalent  to  nouns  in  the  English 
language.  The  entities  in  the  SDT  are  functional 
requirements,  system  missions,  equipment,  tasks, 
courses,  training  media,  and  personnel. 

o  Subentities  -  Lower  level  system  elements. 

Subentities  are  linked  to  entities  in  a 
hierarchical  fashion.  For  example,  "task 

conditions"  are  subentities  of  "tasks". 

o  Attributes  -  Descriptors  that  delimit  or  specify 

important  properties  of  entities.  Each  attribute 
has  a  set  of  values  associated  with  it.  Attributes 

are  used  to  describe  both  entities  and 

subentities.  For  example,  one  attribute  for  the 

entity  task  is  "task  frequency." 
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o  Pointer  Variables  -  Variables  used  to  specify  the 
relationship  which  exist  between  different 
entities,  between  entities  and  subentities  and 
between  entities,  subentities,  and  attributes.  The 
relationships  specified  by  the  pointer  variables 
determine  the  SDT  data  structure.  (The  SDT  uses 
elements  of  both  hierarchical  and  relational  data 
base  structures.) 

I. 5. 1.3  SDT  Configuration 

When  fully  implemented,  the  SDT  will  utilize  a  distributed 
processing  architecture  (see  Figure  1-3).  A  centralized 
data  base  for  each  weapon  system  (or  weapon  system 
alternative)  will  be  stored  on  a  mainframe  computer.  At 
periodic  intervals,  users  will  transfer  a  copy  of  the  data 
base  from  the  mainframe  to  a  local  microcomputer.  Once  on 
the  micro,  users  can  perform  standard  data  base  management 
functions  (input,  output,  modify).  Thus,  all  major  data 
base  management  functions  can  be  performed  independently. 
Once  users  have  completed  their  activities,  they  can 
transfer  the  updated  version  to  the  mainframe.  A  detailed 
"audit  trail"  will  be  kept  for  each  weapon  system  so  that 
users  can  systematically  track  and  assess  system  changes. 
The  current  version  of  the  SDT  is  designed  to  operate  on  (1) 
an  APPLE  III  microcomputer  with  a  modem,  printer,  monitor, 
floppy  disk  drive,  and  a  5  megabyte  hard  disk  and  (2)  a 
Honeywell  DPS  8/32  mainframe  computer.  The  mainframe 
computer  is  actually  only  needed  when  there  is  more  than  one 
user  and  these  users  are  located  at  several  different 
sites.  The  SDT  has  built-in  security  features  which  allow 
the  SDT  manager  to  restrict  data  input,  modify,  and  output 
capabilities  to  a  limited  set  of  users. 
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Figure  1-3.  ETES  Architecture. 


I. 5. 1.4  SDT  Operation 


r 

ia 

B 


N 


To  provide  a  user-friendly  interface,  dialogue  on  the  micro 
computer  relies  on  menu  selection  techniques  for  data  item 
and  command  selection  and  data  output.  Form-filling  and 
question  and  answer  dialogue  techniques  are  used  for  data 
input.  A  "help"  key  is  provided  to  allow  users  to  obtain 
guidance  on  all  SDT  commands  and  data  elements.  This  "help" 
capability  can  be  activated  at  any  time  during  the  operation 
of  the  SDT  by  pressing  the  Escape  key  on  the  APPLE  keyboard. 

1.5. 1.5  Generation  of  Data  for  SDT 

In  order  to  provide  a  capability  for  early  training 
requirements  estimation,  the  SDT  will  be  used  to  describe 
system  elements  during  the  earliest  phases  of  the 
acquisition  process.  To  generate  data  during  these  early 
phases,  comparability  analysis  procedures,  which  are  part  of 
the  ETES  Training  Estimation  Aids/Procedures,  will  be 
employed. 

During  the  early  phases  of  the  system  acquisition,  when  only 
information  on  a  system's  functional  requirements  is 
available,  comparability  analysis  techniques  will  be  used  to 
identify  existing  subsystems  which  are  similar  to  those  of 
the  new  system.  Historical  data  for  these  subsystems  will 
then  be  collected  and  modified  to  (1)  meet  the  differential 
characteristics  of  the  new  system  and  (2)  correct  any 
inherent  deficiencies  in  the  historical  data  base.  By 
utilizing  design  and  task  data  from  comparable,  existing 
systems,  meaningful  early  estimates  of  training  requirements 
can  be  made  when  only  functional  information  on  the 
projected  system  is  available  (see  Figure  1-4).  Later,  as 
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-  Estimated  via  Comparability  Analysis 
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actual  design  concepts  are  developed,  comparability  analysis 
can  be  used  to  develop  estimates  of  tasks  and  training 
program  elements.  Still  later,  when  the  actual  system  tasks 
are  available,  only  the  training  program  elements  must  be 
estimated. 

Thus,  by  adding  comparability  analysis  procedures  to  SDT 
data  base  management  capabilities,  the  SDT  will  be  capable 
of  (1)  describing  alternative  system  concepts  during  the 
earliest  phases  of  the  acquisition  process,  (2)  describing 
projected  system  elements,  (3)  relating  alternative  system 
concepts  to  a  common  framework  so  that  meaningful 
comparisons  can  be  made,  and  (4)  refining  system  information 
as  more  accurate  and  more  detailed  data  are  developed. 

I. 5. 1.6  Application  of  SDT  in  the  System  Acquisition 
Process 

In  its  initial  application  to  a  particular  weapon  system 
development  process,  the  SDT  can  be  used  to  describe  the 
system  functional  requirements  which  are  generated  during 
functional  analysis.  These  requirements  specify  the 
functions  which  must  be  performed  if  the  system  is  to 
satisfy  its  designated  mission  need.  The  SDT  can  be 
employed  in  a  functional  analysis  as  soon  as  the  need  for  a 
particular  system  has  been  specified.  Formally,  this 
activity  occurs  after  the  approval  of  the  requirements 
document  at  Milestone  0,  which  initiates  the  Conceptual 
phase  of  the  acquisition  process.  However,  the  SDT  could 
probably  be  used  to  describe  functional  requirements  even 
prior  to  Milestone  0  if  alternative  system  concepts  were 
identified  earlier. 
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Once  the  functional  requirements  for  a  system  have  been 
developed  and  described  via  the  SDT,  the  SDT  can  be  used  to 
generate  and  describe  system  designs.  These  designs  specify 
possible  mechanisms  for  performing  the  desired  functions. 
These  mechanisms  include  equipment,  personnel,  and  software. 
Once  developed,  the  system  designs  can  also  be  described 
with  the  SDT. 

Once  the  mechanisms  for  accomplishing  the  functions  have 
been  identified  in  the  design  concepts,  the  human  tasks 
which  must  be  performed  to  utilize  the  system  designs  can  be 
specified.  -These  tasks,  which  are  the  key  building  blocks  of 
training  development,  can  be  documented  in  the  SDT.  After 
the  tasks  are  identified  and  specified  in  the  SDT,  training 
estimation  aids  and  procedures  can  be  used  to  determine 
training  program  elements,  estimate  training  resources,  and 
develop  training  products.  The  resulting  training  program 
and  its  associated  resources  can  then  be  documented  in  the 
SDT. 

The  SDT,  like  the  other  components  of  ETES,  is  primarily 
designed  for  applications  during  the  Concept  Exploration 
phase  of  the  acquisition  process,  which  runs  from  Milestone 
0  to  Milestone  1.  However,  the  ETES  can  also  be  used  during 
subsequent  phases  of  the  acquisition  process.  The  primary 
uses  of  ETES  during  these  later  phases  are  to  (1)  make  more 
detailed  estimates  of  task  and  training  resource  require¬ 
ments,  (2)  determine  the  impact  of  subsequent  design  changes 
on  task  and  training  requirements  via  the  data  base  manage¬ 
ment  capabilities  of  the  SDT,  and  (3)  conduct  trade-off 
studies  of  proposed  solutions  to  identified  training 
problems . 
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1.5.2  Training  Estimation  Aids/Procedures  (TEAP) 

The  Training  Estimations  Aids/Procedures  are  an  integrated 
set  of  procedures  and  automated  aids  for  performing  six  key 
early  training  estimation  functions:  (1)  Functional  Require¬ 
ments  Analysis  -  Systematic  description  of  the  functions 
which  the  system  must  perform  and,  where  necessary,  estima¬ 
tion  of  the  hardware/software  design  concepts  needed  to 
achieve  these  functions,  (2)  Task  Generation  -  identifica¬ 
tion  of  the  tasks  required  to  operate  or  maintain  the 
system,  (3)  Training  Program  Estimation  -  estimation  of 
where  and  how  the  tasks  should  be  trained,  (4)  Training 
Resource  Estimation  -  estimation  of  the  training  resources 
needed  to  implement  the  training  program,  (5)  Training  Cost 
Estimation,  and  (6)  Training  Ef f iciency/"Ef feet iveness" 
Estimation  -  estimation  of  the  adequacy  with  which  the 
training  program  can  be  expected  to  train  personnel. 

A  listing  of  the  ETES  Training  Estimation  Aids  and 
procedures  is  contained  in  Figure  1-5. 

The  TEAP  includes  new  techniques  as  well  as  existing 
methodologies  from  other  ARI  research  projects  such  as  the 
Training  Efficiency  Estimation  Model  (Jorgensen,  Kubala,  and 
Atlas,  1981),  recent  work  on  cost  and  training  effectiveness 
analysis  by  Dawdy,  Chapman,  and  Frederickson  (1981),  and  the 
Hardware/Procurement  -  Military  Manpower  (HARDMAN)  method¬ 
ology  (O'Brien,  1983;  Mannle,  1981).  These  existing  method¬ 
ologies  were  modified  to  meet  the  specific  requirements  of 
early  training  estimation.  Brief  descriptions  of  the 
Training  Estimation  Aids  and  Procedures  follow. 


I. 5. 2.1  Functional  Requirements  Analysis 

In  order  to  estimate  training  requirements  for  a  new  system, 
the  training  developer  must  have  information  on  the  system's 
hardware/software  design,  and  its  functional  requirements. 
Unfortunately,  this  information  is  often  not  available  to 
training  analysts  during  the  early  phases  of  the  acquisition 
process.  Consequently,  it  is  often  the  case  that  no 
systematic  estimates  of  training  requirements  are  developed 
during  these  phases.  The  TEAP  is  designed  to  assist  the 
training  analyst  in  overcoming  these  problems  by  providing 
systematic  techniques  for  estimating  the  system's  functional 
requirements  and  hardware/software  design  during  the  early 
phases  of  system  acquisition.  The  TEAP  procedures  for 
estimating  system  functions,  and  hardware/software  design 
are  at  a  general  level.  They  are  designed  to  provide  the 
minimum  amount  of  information  needed  for  early  training 
estimation.  As  the  development  of  the  system  progresses, 
and  actual  information  on  system  functional  requirements  and 
hardware/software  design  are  developed,  these  system 
elements  no  longer  need  to  be  estimated  by  ETES  procedures 
but  can  be  obtained  directly  from  the  combat  developer 
(functional  requirements)  or  materiel  developer  (hardware/ 
software  design  and  manpower  requirements). 

ETES  functional  requirements  analysis  procedures  provide  a 
description  of  the  information  which  should  be  generated 
during  the  functional  requirements  analysis  and  the  steps 
that  one  must  go  through  to  generate  these  elements. 

Early  estimates  of  system  hardware/software  design  are 
generated  via  comparability  analysis.  Quite  simply, 
comparabili ty  analysis  involves  (1)  identifying  comparable 


existing  systems,  (2)  collecting  data  on  the  comparable 
systems,  and  (3)  modifying  these  data  to  reflect  the 

differences  between  the  comparable  existing  system(s)  and 
the  new  system. 

The  most  recent  version  of  the  DoD  standard  on  Logistics 
Support  Analysis  ( LSA)  ( MIL-STD-1 388 )  identifies  compar¬ 
ability  analysis  as  the  preferred  method  for  estimating  key 
system  elements  during  the  early  phases  of  the  acquisition 
process. 

Identification  of  comparable  equipment  in  ETES  takes  place 
in  three  major  phases:  (1)  identification  of  the 

Predecessor  equipment  system(s),  (2)  identification  of  the 
Baseline  Comparison  System  equipment  system,  and  (3) 

identification  of  the  New  equipment  system.  3  The  BCS  is 
constructed  by  (a)  adding  subsystems  to  the  Predecessor 
subsystem  to  reflect  additional  capabilities  required  in  the 
New  system  (i.e.,  the  system  under  development),  and  (b) 
subtracting  subsystems  to  reflect  capabilities  no  longer 
required  in  the  New  system.  The  BCS  provides  a  baseline  for 
comparing  the  New  system  training  requirements  to  other 
similar  systems.  The  BCS  concept  is  directly  congruent 
with  the  new  version  of  MIL-STD-1388  which  specifically 

calls  for  the  construction  of  a  BCS  during  the  early  phases 

of  the  acquisition  process. 


The  procedures  described  in  this  section  were  derived 
from  the  HARDMAN  Methodology.  A  description  of  the  HARDMAN 
Methodology  is  presented  in  Appendix  D. 
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I. 5. 2. 2  Task  Generation  Procedures 

In  the  present  version  of  ETES,  comparability  analysis  is 
used  as  the  principal  means  of  estimating  task  requirements. 
In  this  approach,  task  data  for  the  comparable  existing 
system(s)  are  collected  and  modified  to  reflect  the 
differences  in  design  and/or  employment  between  the  New  and 
comparable  system. 

The  task  generation  process  begins  with  the  collection  of 
task  data  for  the  equipments  and  MOSs  in  the  BCS  system.  The 
BCS  is  composed  of  the  existing  subsystems  ( including  the 
Predecessor  subsystems)  which  come  closest  to  meeting  the 
New  system  functional  requirements.  The  primary  data  source 
for  existing  task  data  is  the  Soldier's  Manual  since  it,  by 
definition,  contains  all  tasks  associated  with  an  MOS.  The 
task  data  from  existing  systems  are  updated  to  reflect  any 
equipment  or  doctrinal  changes  that  were  made  since  the 
source  was  published. 

Once  the  BCS  task  data  has  been  collected  and  updated,  this 
data  is  examined  and  compared  to  the  New  system  hardware/ 
software  design  and  functional  requirements.  BCS  tasks 
associated  with  equipments  or  functions  which  are  no  longer 
needed  are  eliminated.  New  tasks  are  added  to  reflect  new 
equipment  or  functions.  Descriptions  of  new  tasks  are 
developed  in  accordance  with  existing  Instructional  Systems 
Development  (ISD)  guidelines. 

Identification  of  New  system  operator  tasks  is  facilitated 
by  examining  the  subsystem  functions  identified  during  the 
functional  requirements  analysis. 


By  adding  and  deleting  BCS  tasks,  a  New  system  task  list  can 
be  constructed.  The  reason  for  each  deletion/addition  is 
documented  using  a  systematic  set  of  modification  codes. 

It  is  possible  that  a  New  subsystem  may  require  the  same 
task  as  a  BCS  subsystem  but  that  the  essential  character¬ 
istics  of  this  task  must  be  changed  to  reflect  the  New 
system  requirements.  For  example,  the  same  task  may  be 
required  for  both  the  BCS  and  New  system  but  the  frequency 
with  which  the  task  is  performed  may  differ. 

To  account  for  these  changes  to  essential  task  character¬ 
istics,  all  tasks  with  major  modifications  are  identified 
and  the  reason  for  each  task  modification  is  documented. 

Once  tasks  have  been  identified,  their  conditions  and 
standards  can  be  developed  using  existing  ISD  procedures. 
These  same  procedures  are  used  to  assess  the  adequacy  of  any 
new  task  descriptions  which  are  generated  during  compar¬ 
ability  analysis. 

I. 5. 2.3  Training  Program  Estimation4 

Algorithms  are  provided  to  assist  the  training  analyst  in 
determining  the  tasks  to  be  trained  and  assigning  these 
tasks  to  training  settings.  To  provide  input  to  these 
algorithms,  tasks  are  rated  on  a  series  of  task 
characteristics  (e.a.,  frequency,  learning  difficulty). 
Several  different  algorithms  are  provided  to  meet  the  needs 
of  different  phases  of  the  acquisition  process. 


*  The  current  version  of  ETES  only  contains  procedures 
for  estimating  training  programs  for  individual 
institutional  training  courses. 
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Quasi-programs  of  instruction  (QPOI)  are  constructed  by  (1) 
modifying  or  deleting  modules  from  existing  courses  to 
reflect  the  task  deletions  and  modifications  made  during 
task  generation  and  (2)  adding  modules  to  reflect  the  unique 
requirements  of  the  New  system.  As  part  of  the  QPOI 
construction  process,  the  instructional  methods  and 
curriculum  hours  which  must  be  devoted  to  each  module  are 
determined. 

Media  for  the  training  program  are  selected  by  the 
application  of  an  automated  aid,  the  Media  Selection  and 
Efficiency  Estimation  Program.  This  program  is  an  extension 
of  the  Training  Efficiency  Estimation  Model  (TEEM)  produced 
by  Jorgensen  et  al.  (1981).  The  Media  Selection  and 
Efficiency  Estimation  Program  significantly  expands  the 
capabilities  of  TEEM  by  recasting  media  selection  as  a 
dynamic  programming  problem  and  automating  these  procedures 
on  the  Apple  III  microcomputer.  This  automated  program 
permits  the  user  to  employ  the  SDT  to  input  and  store  the 
data  needed  to  feed  the  media  selection  procedures.  Using 
dynamic  programming  techniques,  the  program  can  assign  tasks 
to  media  in  a  manner  that  optimizes  efficiency,  relative 
cost,  or  efficiency  and  relative  cost. 

Efficiency  is  determined  by  comparing  the  stimulus, 
response,  and  feedback  characteristics  of  the  individual 
task  to  the  stimulus,  response,  and  feedback  characteristics 
of  potential  media  categories.  More  specifically,  a  score 
is  calculated  which  describes  the  match  between  media  and 
task  characteristics.  Efficiency  for  each  task-media 

combination  is  calculated  by  dividing  this  score  by  the 
maximum  match  that  may  be  achieved  for  the  task. 
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Total  efficiency  -for  a  set  of  tasks  is  determined  by 
aggregating  the  efficiency  score  for  individual  tasks.  An 
additional  efficiency  measure  can  be  calculated  by  weighting 
the  efficiency  of  each  task  by  the  task  criticality  score. 
This  task  criticality  score  is  calculated  by  aggregating  the 
task  factors  typically  used  in  selecting  tasks  for  training 
(e.g.,  task  frequency,  percent  members  performing,  task 
delay  tolerance,  etc.). 

1. 5. 2. 4  Estimation  of  Training  Resources 

These  procedures  estimate  the  training  resources  needed  to 
implement  the  training  program.  The  training  resources 
encompassed  by  these  procedures  include  (a)  the  number  of 
students  to  be  trained,  (b)  number  of  instructors  and 
support  personnel,  (c)  facilities  requirements,  (d)  training 
device  and  training  equipment  requirements,  and  (e) 
ammunition  requirements.  Included  among  the  procedures  are 
techniques  for  using  off-the-shelf  automated  worksheet 
software  (e.g.,  VisiCalc)  for  storing  and  applying  several 
of  the  resource  determination  algorithms. 

1. 5. 2.5  Estimation  of  Training  Costs 

These  procedures  estimate  the  costs  of  the  resource 
requirements  and  aggregate  these  costs  into  a  total  course 
cost.  The  procedures  include  techniques  for  using  off-the- 
shelf  automated  worksheet  software  to  calculate  course 
costs.  The  procedures  also  describe  how  to  use  modified  data 
from  comparable  existing  courses  to  assist  in  the  cost 
estimation  process. 
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I. 5. 2. 6  Training  Efficiency/Effectiveness  Estimation 

Procedures  are  provided  for  determining  the  training 
efficiency  of  selected  aspects  of  the  training  program 
(e.g.,  media).  Training  efficiency  is  defined  as  a  measure 
of  the  extent  to  which  the  characteristics  of  the  training 
program  element  match  the  task  characteristics  of  the  New 
system.  For  example,  media  training  efficiency  is 
determined  by  comparing  the  stimulus,  response,  and  feedback 
characteristics  of  the  tasks  to  the  stimulus,  response,  and 
feedback  characteristics  of  potential  media  categories.  An 
additional  variant  of  this  efficiency  measure  may  be 
obtained  by  weighting  each  task  by  its  "criticality"  where 
criticality  is  determined  by  aggregating  weighted  scores  of 
key  task  characteristics  (e.g.,  frequency,  consequences  of 
inadequate  performance). 

Training  "effectiveness'*  is  determined  by  estimates  obtained 
from  subject  matter  experts.  More  specifically,  a  group  of 
subject  matter  experts  is  presented  with  the  following 
information  for  each  task:  (a)  the  target  population 
description  of  the  personnel  who  will  perform  the  task,  (b) 
a  description  of  the  task  including  its  associated 
conditions,  job  performance  standards,  and  general  skills 
and  knowledges,  (c)  a  description  of  the  training  program  or 
training  program  elements  (e.g.,  course  length,  methods, 
media)  that  will  be  used  to  train  the  task,  and  (d)  the 
criterion  which  must  be  achieved  for  the  task  (if  different 
from  the  job  performance  measure).  Each  subject  matter 
expert  is  then  asked  to  estimate  the  expected  percentage  of 


soldiers  in  the  target  population  who  will  pass  the 
criterion  given  that  training  program.^ 

It  should  be  noted  that  the  ETES  TEAP  is  designed  to 
estimate  training  requirements  for  a  New  system.  The  TEAP 
is  not  designed  to  provide  techniques  for  actually 
developing  instructional  materials  or  programs  (e. g.  , 
instructor  packages,  student  packages,  training  literature, 
training  devices).  Thus,  the  TEAP  will  assist  training 
analysts  in  determining  what  instructional 
materials/programs  should  be  produced,  what  the  content  of 
these  materials/programs  should  be,  and  in  estimating  the 
cost  of  these  materials/programs. 

This  focus  on  training  requirements  estimation  rather  than 
on  training  product  development  is  one  of  the  unique 
characteristics  of  the  TEAP.  This  focus  on  training  require¬ 
ments  distinguishes  it  from  other  existing  training  develop¬ 
ment  methodologies  such  as  the  Army's  Instructional  Systems 
Development  (ISD)  process  and  the  Army  Research  Institute's 
Training  Developer's  Decision  Aid  (Hawley,  1979). 

The  training  requirements  produced  by  the  TEAP  provide 
front-end  information  needed  for  the  early  planning  and 
analysis  of  training  programs.  The  training  requirements 
information  produced  by  the  TEAP  provides  the  foundation  for 
the  actual  construction  of  training  products  in  ISD  and 


It  is  recognized  that  this  measure  of  effectiveness, 
may  differ  from  other  conceptualizations  of  training 
effectiveness.  It  is  also  recognized  that  this  approach 
only  provides  a  gross  measure  of  "effectiveness"  that  should 
only  be  used  during  the  earliest  phases  of  the  acquisition 
process . 
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other  related  methodologies.  Techniques  and  methodologies 
which  can  be  used  to  develop  training  products  are  provided 
in  TRADOC  Pam  350-30,  Schulz  and  Farrell  (1980),  Hawley 
(1979),  and  Fink  (1981).  An  overview  of  the  relationship 
between  ISD  and  ETES  is  provided  in  Figure  1-6. 

1.5.3  Evaluative  Technology 

The  Evaluative  Technology  is  an  integrated  set  of  procedures 
and  automated  tools  for  (1)  developing  f igures-of-merit  for 
assessing  the  integrated  impacts  of  the  training  require¬ 
ments  developed  in  the  Training  Estimation  Aids/Procedures, 
(2)  identifying  potential  problem  areas  for  system  training 
and  the  likely  sources  of  these  problems,  (3)  identifying/ 
evaluating  training  problems,  (4)  developing  training- 
related  input  to  key  acquisition  documents,  and  (5) 
determining/evaluating  training  development  schedules  (see 
Figure  1-7).  A  summary  of  the  procedures  in  each  of  these 
areas  is  provided  below. 

I. 5. 3.1  Development  of  Figures-of-Meri t 

Procedures  are  included  for  identifying  f igures-of-mer i t 
which  summarize  the  essential  features  of  the  training 
requirements.  Eight  potential  training  f igures-of-merit  are 
utilized  including  (1)  cost,  (2)  training  efficiency,  (3) 
training  "effectiveness",  (4)  congruence  with  training 
development  guidelines,  (5)  congruence  with  program 
requirements,  (6)  training  complexity,  (7)  feasibility,  and 
(8)  summary  evaluation  score  which  aggregates  the  scores  on 
the  most  critical  f igures-of-merit  (cost  effectiveness,  and 
complexity).  This  summary  evaluation  score  provides  a 
global  measure  of  the  "goodness"  of  a  training  program. 
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FIGURE  1-6.  RELATIONSHIP  BETWEEN  ISD  AND  ETES 
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Figure  1-7  ETES  Evaluative  Technology 


1. 5.3.2  Identification  of  Problem  Areas 

Procedures  are  included  for  identifying  training  problem 
areas.  These  problem  areas  consist  of  the  Army  Military 
Occupational  Specialties  (MOSs)  which  have  high  figure-of- 
merit  scores  relative  to  (1)  the  Predecessor  system,  and/or 

(2)  the  other  MOSs  in  the  New  system.  Procedures  are  also 
provided  for  identifying  the  courses,  equipments,  and  tasks 
which  contribute  to  the  high  f igures-of-mer i t  and 
identifying  the  training  program  elements  which  are  likely 
causes  of  the  high  f igure-of-mer i t  scores. 

1.5. 3. 3  Identification  and  Evaluation  of  Alternatives 

Guidelines  are  included  for  (1)  identifying  training 
alternatives  which  can  address  the  training  problems,  (2) 
evaluating  the  training  alternatives  through  selected 
reapplication  of  TEAP  and  Evaluative  Technology  procedures, 

(3)  conducting  sensitivity  analyses  of  key  parameters,  and 

(4)  assessing  the  impact  of  non-training  system  changes 
(e.g.,  hardware/sof tware  changes)  on  training. 

1. 5. 3. 4  Development  and  Evaluation  of  Training 
Schedules/Plans 

Construction  of  training  development  schedules  for  emerging 
systems  is  a  difficult  process.  Over  100  developmental 
events  are  listed  in  TRADOC  Reg  351-9,  the  Army  regulation 
governing  training  plan  development.  The  sequential 

relationships  among  these  events  are  complex  and  are  not 
described  in  any  systematic  and  integrated  manner  in  TRADOC 
Reg  351-9.  To  assist  users  in  developing  training 

schedules,  procedures  are  provided  for  using  automated,  off- 


the-shelf  scheduling  software  (e.g.,  Vi siSchedule )  to  track 
and  monitor  the  training  development  schedule.  By  using 
this  software,  the  training  developer  can  quickly  and 
efficiently  respond  to  changes  in  the  training  development 
schedule.  Use  of  off-the-shelf  scheduling  software  is 
facilitated  by  the  inclusion  of  an  input  data  diskette  which 
(a)  describes  the  events  in  the  training  development  process 
(as  specified  in  TRADOC  Reg  351-9),  (2)  describes  the 
temporal/sequential  relationships  among  these  events  and  key 
acquisition  milestones,  and  (3)  lists  the  expected  duration 
of  these  events  for  a  "typical"  major  Army  weapons  system. 
This  data  diskette  significantly  reduces  data  input 
requirements.  In  addition,  it  eliminates  the  need  for  an 
analysis  of  the  complex  sequential  relationships  among 
training  development  events  which  are  either  implicitly  or 
explicitly  specified  in  TRADOC  351-9. 

I. 5. 3. 5  Develop  Inputs  to  Acquisition  Processes  and 
Documents 

Procedures  are  provided  for  using  ETES  products  to  provide 
inputs  to  the  major  Army  acquisition  processes  and  documents 
related  to  early  training  estimation.  These  documents  and 
processes  are  summarized  in  Table  1-2. 

1.6  KEY  CONCEPTS  UNDERLYING  ETES 

This  section  provides  a  description  of  several  of  the  key 
concepts  underlying  the  ETES.  Section  1.6.1  describes 
comparability  analysis,  which  is  a  systematic  set  of 
procedures  for  using  data  from  comparable  existing  systems 
to  estimate  requirements  for  new  systems.  Comparability 
analysis  is  particularly  appropriate  during  the  early  phases 
of  the  acquisition  process. 
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Table  1-2.  Major  Army  Acquisition  Processes 
and  Documents  Related  to  Early 
Training  Estimation 

Cost  and  Training  Effectiveness  Analysis  (CTEA) 

Outline  Individual  and  Collective  Training  Plan  (OICTP/ICTP) 
Logistics  Support  Analysis  (LSA) 

Operational  Testing  (OT) 

Training  Device  Requirements,  Documents,  and  Processes 

-  Training  Device  Letter  of  Agreement  (TDLOA) 

-  Training  Device  Requirements  (TDR) 

-  Training  Device  Letter  Requirement  (TDRL) 

-  Training  Device  Study  (TDS) 

New  Equipment  Training 
System  Requirements/Documents 

-  Justification  for  Major  System  New  Start  (JMSNS) 

-  Letter  of  Agreement  (LOA) 

Request  for  Proposal  (RFP)  Development/Evaluation 
Personnel  Documents/Processes 

-  Tentative  Quantitative  and  Qualitative  Personnel 

Requirements  Information  (TQQPRI) 

Integrated  Personnel  Summary  (IPS) 

System  Level  Documents/Processes 

-  Concept  Formulation  Package  (CFP) 

-  Cost  and  Operational  Effectiveness  Analyses  (COEA) 

-  Tradeoff  Determination  (TOD) 

-  Best  Technical  Approach  (BTA) 

-  Decision  Coordinating  Paper  (DCP) 

-  Defense  System  Acquisition  Review  Council  (DSARC) 

-  Ar...y  System  Acquisition  Review  Council  (ASARC) 

Mission  Area  Analysis 


Section  1.6  describes  the  differences  between  steady  state 
and  phased  resource  requirements  estimation.  Section  1.6.3 
describes  the  limitations  of  the  current  ETES. 

1.6.1  Comparability  Analysis 

ETES  is  designed  to  determine  training  requirements  during 
the  earliest  phases  of  the  acquisition  process.  However, 
during  these  early  phases  details i  data  on  system  equipment 
and  key  training  elements  (e.g.,  tasks)  are  typically 
unavailable.  This  poses  an  initial  problem,  because  without 
such  data,  estimates  of  training  requirements  cannot  be 
made.  To  circumvent  this  problem,  comparability  analysis 
can  be  used  to  identify  these  key  system  elements.  Quite 
simply,  comparability  analysis  involves  (1)  identifying 
comparable  existing  systems,  (2)  collecting  data  on  the 
comparable  systems,  and  (3)  modifying  these  data  to  reflect 
the  differences  between  the  comparable  existing  system(s) 
and  the  New  system. 

The  most  recent  version  of  the  DoD  standard  on  Logistics 
Support  Analysis  ( LSA)  ( MI L-STD-1 388 )  identifies  compar¬ 
ability  analysis  as  the  preferred  method  for  estimating  key 
system  elements  during  the  early  phases  of  the  acquisition 
process. 

An  overview  of  the  basic  process  that  is  employed  in  compar¬ 
ability  analysis  procedures  is  outlined  in  Figure  1-8.  The 
process  begins  by  determining  if  the  necessary  system 
element  (e.g..  New  system  equipment  configuration,  tasks) 
has  already  been  developed.  If  this  system  element  does  not 
exist,  a  comparable  existing  system  element  is  identified. 
A  comparable  existing  system  element  is  a  system  element 


Figure  1-8.  Overview  of  Comparability  Analysis  Procedures, 
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which  performs  the  same  function  as  the  New  system  element. 
There  may  be  more  than  one  comparable  system  element.  In 
these  cases,  the  "best"  comparable  existing  system  is 
selected.  The  "best"  comparable  element  is  the  system 
element  which  (1)  has  performance  requirements  of  the  New 
system,  (2)  has  available  data,  and  (3)  has  been  used  in  an 
environment  which  closely  matches  the  scenario,  organiza¬ 
tional,  and  operational  and  support  concepts  of  the  proposed 
New  system. 

Once  the  comparable  existing  system  element  is  identified, 
the  data  associated  with  that  element  is  collected.  This 
data  is  then  modified  to  reflect  any  inherent  inaccuracies 
associated  with  the  data  base(s)  from  which  the  data  was 
collected.  The  data  is  also  adjusted  to  reflect  differences 
between  the  comparable  system  and  the  New  system  in  terms  of 
scenarios  and  operational  concepts.  In  the  final  step  in 
the  comparability  analysis  process,  data  on  the  existing 
system  is  modified  to  reflect  the  differences  between  the 
functional  capabilities  of  the  existing  system  element  and 
the  functional  requirements  of  the  New  system  element. 

The  comparability  analysis  process  outlined  in  Figure  1-8 
may  be  used  to  identify  functional  requirements,  the  system 
hardware/software  design,  system  manpower  requirements,  and 
system  task  and  skill  requirements.  It  may  also  be  used  to 
provide  high  level  estimates  of  system  training  program 
requirements,  training  resources,  and  training  costs. 

The  key  applications  of  comparability  analysis  is  in  the 
identification  of  comparable  equipment  which  occurs  in  TEAP 
Function  2.0. 


A  more  detailed  description  of  the  procedures  used  to 
identify  comparable  equipment  is  presented  in  the  subsection 
which  follows. 

I. 6. 1.1  Identification  of  Comparable  Equipment6 

Identification  of  comparable  equipment  takes  place  in  three 
major  phases:  (1)  identification  of  the  Predecessor 
equipment  system(s),  (2)  identification  of  the  BCS  equipment 
system,  and  (3)  identification  of  the  New  equipment 
system.  The  BCS  system  is  constructed  by  (a)  adding 
subsystems  to  the  Predecessor  subsystem  to  reflect 
additional  capabilities  required  in  the  New  system  (i.e., 
the  system  under  development),  and  (b)  subtracting 
subsystems  to  reflect  capabilities  no  longer  required  in  the 
New  system.  The  BCS  provides  a  baseline  for  comparing  the 
New  system  training  requirements  to  training  requirements 
for  other  similar  systems.  The  BCS  concept  is  directly 
congruent  with  the  new  version  of  MIL-STD-1388  which  speci¬ 
fically  calls  for  the  construction  of  a  BCS  during  the  early 
phases  of  the  acquisition  process. 


Predecessor  Equipment  System(s). 


The  Predecessor 


equipment  system  is  the  system(s)  which  is  currently 
performing  the  mission(s)  which  will  eventually  be  performed 
by  the  New  system.  Thus,  the  Predecessor  equipment  system 
will  be  replaced  by  the  New  system.  By  definition,  the 
Predecessor  equipment  is  currently  in  the  DoD  inventory.  In 
some  cases,  a  Predecessor  equipment  system  may  not  exist. 


b  The  procedures  described  in  this  section  were  derived 
from  the  HARDMAN  Methodology.  A  description  of  the  HARDMAN 
Methodology  is  presented  in  Appendix  D. 


BCS  Equipment  System.  The  BCS  equipment  system  is  a 
notional  design  composed  of  the  existing  subsystems  which 
most  closely  match  the  functional  requirements  of  the  New 
system.  The  BCS  equipment  system  is  designed  to  meet  each 
of  the  general  functional  requirements  required  by  the  New 
system.  However,  since  it  is  composed  solely  of  existing 
subsystems  from  the  DoD  inventory,  it  will  not  fully  meet 
all  of  the  performance  goals  associated  with  the  New  system 
functions. 

In  order  to  be  selected  for  the  BCS  equipment  system,  a 
subsystem  must  meet  the  following  criteria:  (1)  it  must  be 

an  existing  subsystem  currently  in  the  DoD  inventory 
(preferably  in  the  Army  inventory),  (2)  it  must  be  the 
existing  subsystem  which  most  closely  meets  system  perfor¬ 
mance  requirements,  and  (3)  it  must  have  mature  task  data 
(the  latter  set  of  data  is  critical  for  the  training 
requirements  determination). 

New  Equipment  System.  The  new  equipment  system  is  an 
integrated  system  which  is  designed  to  fully  meet  all  system 
performance  requirements.  During  the  early  phases  of  the 
acquisition  process,  prior  to  the  receipt  of  the  con¬ 
tractor's  design  concept,  a  notional  New  equipment  system  is 
constructed  by  (1)  estimating  the  new  technologies  that  must 
be  added  to  their  BCS  equipment  subsystems  so  that  the  new 
system  performance  requirements  can  be  fully  met,  and  (2) 
modifying  the  BCS  equipment  data  to  reflect  the  addition  of 
these  technologies. 

During  the  later  phases  of  the  acquisition  process,  the 
contractor's  design(s)  can  be  used  as  the  New  system(s). 
Thus,  while  there  is  one  BCS  equipment  system,  there  may  be 


I 


several  different  New  equipment  systems  during  the  develop¬ 
ment  of  any  one  weapon  system. 

I. 6. 1.2  Development  of  Predecessor,  BCS,  and  New  Systems 
Training  Subsystem 


r 


Predecessor,  BCS,  and  New  training  subsystems,  (i.e.,  a 
system  which  includes  task  training  programs  and  training 
resources  and  costs)  may  be  constructed  to  correspond  the 
Predecessor,  BCS  and  New  equipment  systems.  The  Predecessor, 
BCS,  and  New  training  subsystems  may  be  defined  as  follows. 

Predecessor  System.  The  predecessor  training  subsystem 
is  composed  of  the  tasks,  and  training  programs,  associated 
with  the  predecessor  equipment  and  the  training  resources 
and  cost  associated  with  these  training  programs. 

Baseline  Comparison  System  (BCS).  The  BCS  training 
subsystem  contains  tasks,  training  programs,  and  training 
resources  and  costs  associated  with  the  BCS  equipment.  Since 
the  BCS  equipment  subsystems  are  existing  subsystems,  the 
training  elements  of  these  subsystems  can  also  be  directly 
obtained  from  existing  data.  However,  BCS  task  and  training 
data  may  be  modified  to  reflect  (a)  any  inherent  inaccur¬ 
acies  in  the  data  bases  from  which  it  is  derived,  and  (b) 
differences  between  the  BCS  subsystem  scenarios  and  opera¬ 
tional  concepts  and  the  New  system  scenarios  and  operational 
concepts  which  may  impact  task  requirements  or  training 
resource  usage. 

New  System.  The  training  subsystem  for  the  New  system 
contains  the  tasks,  training  programs,  and  training 
resources  and  costs  associated  with  the  New  system  equip¬ 
ment.  During  the  early  phases  of  the  acquisition  process, 
New  system  task  requirements  are  derived  by  modifying  the 
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BCS  task  requirements  to  reflect  the  design  differences 
between  the  BCS  and  New  equipment  systems.  During  the  later 
phases  of  the  acquisition  process,  task  data  may  be  supplied 
by  the  contractor ( s )  or  cognizant  government  agencies. 

Once  task  requirements  have  been  determined,  it  is  possible 
to  apply  a  wide  range  of  analytical  techniques  to  determine 
the  training  program  requirements  and  training  resources  and 
costs  for  the  New  system. 

1.6.2  Phased  Versus  Steady-State  Training  Resource 
Requirements 

In  estimating  training  resource  requirements  for  a 
developing  system,  it  is  important  to  distinguish  between 
phased  and  steady-state  resource  requirements.  Phased 
resource  requirements  (e.g.,  NET  resource  requirements)  are 
the  specific  resources  that  are  required  while  the  weapon 
system  is  being  placed  into  the  inventory,  or  while  the 
weapon  system  is  being  removed  from  the  inventory  (the 
latter  case  is  generally  not  relevant  during  system 
development).  Phased  resource  requirements  are  displayed  by 
calendar  year.  In  order  to  determine  these  phased  resource 
requirements  during  system  development,  information  on  the 
number  of  weapon  systems  introduced  into  the  field  in  each 
calendar  year  must  be  available  and/or  estimated. 

Steady-state  resource  requirements  are  the  estimated  average 
yearly  resource  requirements  that  are  needed  to  maintain  the 
weapon  systems  in  the  field  once  all  of  the  systems  have 
been  fielded.  To  estimate  steady-state  resource  require¬ 
ments,  all  that  is  required  is  an  estimate  of  the  total 
number  of  systems  that  will  be  in  the  field  once  system 
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installation  has  been  completed.  Steady-state  requirements 
are  also  presented  in  yearly  units. 

ETES  provides  the  capability  for  estimating  both  steady- 
state  and  phased  MPT  requirements.  This  is  accomplished  by 
first  estimating  the  steady-state  requirements  and  then 
modifying  these  requirements  to  meet  the  specific  phased 
requirements  that  are  required  to  install  the  system.  This 
approach  differs  from  the  usual  approach  of  developing  the 
phased  requirements  first  and  then  modifying  the  phased 
requirements  to  produce  the  steady-state  requirements.  The 
ETES  approach  (steady-state,  then  phased  requirements)  has 
an  obvious  advantage  in  that  it  starts  with  the  end  objec¬ 
tives  (steady-state  requirements)  and  then  identifies  the 
intermediate  products  (phased  requirements)  needed  to  meet 
those  objectives. 

1,6.3  Limitations  of  Current  ETES 

The  current  version  of  ETES  has  the  following  limitations:^ 

o  It  does  not  estimate  training  programs  for 
individual  unit  training  or  collective  training. 

o  It  does  not  estimate  acquisition-related  training 
costs. 


These  limitations  are  expected  to  be  addressed  in  follow- 
on  research  conducted  by  the  Army  Research  Institute.  A 
more  detailed  description  of  the  limitations  of  the  current 
version  of  ETES  is  provided  in  the  ETES  Final  Report. 


o  It  provides  relatively  crude  estimates  of  training 
effectiveness  which  are  only  appropriate  for  the 
very  earliest  phases  of  the  acquisition  process. 

1.7  GUIDELINES  FOR  MANAGING  ETES  APPLICATION 

The  section  describes  general  guidelines  for  managing  an 
ETES  application.  Guidelines  are  provided  in  two  key 
areas:  data  management  and  study  plan  construction. 

1.7.1  ETES  Data  Management 

To  successfully  manage  the  data  associated  with  an 
application  of  ETES  the  user  must;  (1)  track  and  monitor  all 
input  data  and  data  products  associated  with  each  ETES 
procedure,  (2)  establish  and  maintain  an  audit  trail  of  the 
products  produced  during  each  procedure,  and  (3)  store 
relevant  input  and  output  data  in  the  System  Description 
Technology  ( SDT )  data  base  management  system. 

An  overview  of  the  elements  in  the  ETES  data  management 
process  is  provided  in  Figure  1-9.  A  more  detailed 
description  of  these  elements  is  provided  below. 

Raw  Input  Data.  The  raw  input  data  needed  to  implement 
ETES  procedures  may  come  in  a  variety  of  different  forms 
(including  hard-copy  documents,  interviews,  mag.  ''tic  tapes, 
magnetic  discs,  and  direct  on-line  data  transmissions). 

Data  Source  Index.  The  data  source  index  is  a 
systematic  table  describing  the  source  from  which  each  of 
the  raw  input  data  were  obtained.  The  data  sources  in  the 
data  source  index  are  grouped  according  to  major  functional 
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Data  Management 


data  categories.  An  example  of  a  page  from  a  data  source 
index  is  presented  in  Table  1-3.  A  complete  data  source 
index  listing  most  of  the  major  data  sources  that  will  be 
employed  in  an  ETES  application  is  provided  in  Appendix  F. 
ETES  data  is  obtained  from  both  system-specific  sources 
(i.e.,  data  bases  directly  related  to  the  system  under 
investigation)  and  nonsystem-specific  sources  (i.e.,  data 
bases  containing  data  on  a  wide  range  of  systems).  The  data 
source  index  can  be  manual.  However,  it  should  be  automated 
to  facilitate  construction  and  subsequent  update. 

Library  Index.  The  library  index  is  a  systematic 
document  describing  and  cataloging  (a)  each  raw  input  data 
element  and  (b)  each  ETES  worksheet.  The  worksheets 
describe  the  intermediate  products  of  ETES  analytical 
procedures.  The  descriptions  in  the  library  index  include 
standard  bibliographic  information  on  the  input  data.  They 
also  indicate  exactly  where  the  input  data  or  worksheet  is 
currently  located  (e.g.,  who  has  the  document,  worksheet, 
etc.).  Table  1-4  also  provides  an  excerpt  from  a  page  in  a 
library  index. 

Extraction  Procedures.  The  raw  data  from  existing  Army 
data  sources  may  not  always  be  in  a  form  which  is  appro¬ 
priate  for  ETES  procedures.  In  these  cases,  systematic 
procedures  for  extracting  the  required  data  elements  must  be 
developed  as  part  of  the  data  management  process.  The 
specific  extraction  procedures  that  are  employed  will  vary 
depending  on  the  data  base  and  the  particular  focus  of  the 
ETES  application.  Specific  details  on  data  extraction 
procedures  must  be  obtained  from  the  organization  control¬ 
ling  the  data  base.  The  organi zat ions  associated  with  the 
major  data  sources  are  listed  in  Appendix  E. 
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Table  1-4.  Example  Page  from  Library  Index 


Date  Entered:  ETES  Procedure: 


Data  Input  No.:  Data  Output  No.: 


Worksheet  No.: 


Responsible  Agency 


Reference: 


Document  Type: 
Journal: 
Article: 

Book: 

Report: 
Other— Print: 


Tape: 

Disc: 

Comp.  Cards: 

Direct  Comp.  Link: 
Other: 


SDT .  The  SDT  provides  a  system-specific  data  base  for 
describing  the  basic  system  elements  which  are  required  to 
apply  ETES  procedures.  The  system-specific  data  base 
includes  both  initial  input  data  and  output  products  from 
ETES  procedures,  as  well  as  changes  to  these  elemencs  which 
are  made  during  subsequent  applications  of  the  procedures. 
The  SDT  contains  the  key  data  elements  needed  to  character¬ 
ize  a  developing  system  with  an  emphasis  on  the  task  and 
training  components  of  the  system.  The  SDT  is  congruent 
with  the  system  description  provided  in  the  Logistics 
Support  Analysis  Record. 

Audit  Trail  Procedures.  The  ETES  audit  trail 
procedures  provide  a  systematic  mechanism  for  (a)  tracking 
the  development  of  training  requirements  and  (b)  monitoring 
changes  to  the  training  requirements.  The  audit  trail 
procedures  permit  another  analyst  to  replicate  and/or 
validate  the  study  results.  The  audit  trail  procedures 
document  the  data,  procedures,  and  assumptions  underlying 
each  step  of  the  methodology. 

An  overview  of  the  audit  trail  procedures  is  outlined  in 
Figure  1-10.  First,  as  data  is  obtained,  its  source  and 
major  features  must  be  systematically  documented  in  the  data 
source  index  and  library  index.  Furthermore,  as  data  is 
distributed  among  analysts,  its  location  must  be  tracked  in 
the  library  index. 

After  being  collected  and  entered  in  the  SDT,  the  data  will 
be  analyzed  in  the  ETES  procedures.  As  each  procedure  is 
completed,  the  analytical  steps  in  these  procedures  should 
be  documented  in  the  ETES  worksheets.  A  description  of  each 
worksheet  and  its  current  location  should  also  be  entered 


Figure  1-10  Overview  of  Audit  Trail  Procedures 


< 


As  the  system  progresses,  there  will  be  frequent  changes  to 
its  data  elements.  These  changes  should  be  documented  in 
the  SDT. 


Worksheets .  Worksheets  are  forms  designed  to  describe 
the  intermediary  results  of  the  analytical  steps  and 
algorithms  in  the  ETES  procedures.  An  example  of  a 

worksheet  is  presented  in  Table  1-5.  To  assist  the  analyst 
in  monitoring  the  audit  trail,  each  worksheet  should  be 
assigned  a  unique  identification  number  describing  (1)  the 
ETES  function  in  which  it  is  applied,  (2)  the  equipment  with 
which  it  is  associated,  (3)  the  MOS/ASI/skil 1 
level/duty  position  with  which  it  is  associated,  and/or  (4) 
the  course  with  which  it  is  associated.  A  listing  of  the 
ETES  worksheets  is  presented  in  Table  1-6. 

1.7.2  ETES  Study  Plan  Development 


Prior  to  the  application  of  the  ETES  TEAP  and  Evaluative 
Technology,  a  systematic  study  plan  must  be  established.  A 
potential  outline  for  an  ETES  study  plan  is  provided  in 
Table  1-7.  There  are  four  major  questions  which  have  to  be 
asked  during  the  development  of  a  study  plan: 

(1)  What  resources  will  be  available  to  conduct  the 
ETES  application? 

(2)  What  is  the  scope  of  the  study  (i.e.,  what  ETES 
procedures  will  be  applied)? 

(3)  What  is  the  focus  of  the  study  (i.e.  on  what 
elements  of  the  system  will  the  ETES  procedures  be 
applied ) ? 
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Table  1-6.  List  of  ETES  Worksheets. 


ETES  Study  Plan  Outline  (OUT) 

Acquisition  Information  (ACQ) 

Mission  Profile  (MIS) 

System  Performance  Specification  (PRF) 

Organizational,  Operational,  and  Support  Concepts  (OSC) 
Operational  Environment  (ENV) 

System  Performance  Specification  (PRF) 

Generic  System  (GEN) 

Function  Sequence  (FSQ) 

Baseline  Comparison  Equipment  Selection  (EQP) 

BCS  Equipment  Description  (BCS) 

Baseline  Comparison  Performance  Shortfall  (PSH) 

Design  Difference  (DD) 

New  System  Equipment  Description  (EQP) 

Candidate  MOS  (CND) 

BCS  Task  Association  (BCST) 

New  System  Task  Generation  (NTA) 

DIF  Data  Collection  (DIF) 

Task  Data  (PSYCRIT) 

Select  Tasks  for  Training  (TSEL) 

Media  Assignment  (MAS) 

Media  Selection  Planning  (MSEL) 

Data  Summary  for  Training  Setting  Assignment  (SUMSET) 
Skill  and  Knowledge  (SKILL) 

Target  Population  (POP) 

Task/Skill  Sequence  (SEQ) 

QPOI  -  Parts  1  &  2  (QPOI-1,  QPOI-2) 

Course  Modification  (MOD) 

Training  Path  (PATH) 

Number  of  Students  —  Steady-State  (NSSS) 

Phased  Course  Input  Requirements  (PHASE) 

Number  of  Students  to  be  Trained  (NSTUD) 

Number  of  Instructors  (N1NS) 

Determine  Training  Facilities  (DFAC) 

Facilities  Requirements  (NFAC) 

Determine  Number  of  Training  Devices/Equipments  (TDEV) 
Training  Device/Equipment  (DEEQ) 


Table  1-6.  List  of  ETES  Worksheets  (Continued). 


Determine  Number  of  Media  and  Instructional  Aids  (NMED) 
Number  of  Media/Aids  (MEDSUM) 

Ammunition  Requirements  (AMREQ) 

Ammunition  Requirements  Summary  Sheet  (AMSUM) 
Operating  and  Support  Plan  (OPSP) 

Media  Usage  Rate  (MUS) 

Individual  Course  Cost  (I COST) 

Course  Aggregation  (ACOST) 

Training  Summary  (TVICE) 

Method  Rating  (METH) 

Task  Rating  for  Instructional  Methods  (TAMOD) 

Calculate  Method  Efficiency  (METHEFF) 

Calculate  Overall  Effectiveness  Score  (EFF) 


TABLE  1-7  ETES  STUDY  PLAN  OUTLINE  (OUT) 


I.  Available  Resources  and  Costs 


Labor 


Category  1 _ 

Category  2 _ 

Category  3 _ 

Category  4 _ 

Category  5 _ 

Total 

*in  person-months 


Costs 

Category  1 _ 

Category  2 _ 

Category  3 _ 

Category  4 _ 

Category  5 _ 

Total 

Computer  Resources 

Category  1 _ 

Category  2 _ 

Category  3 _ _____ 

Category  4 

Category  5 _ 

Total 
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Table  1-7  Study  Plan  (continued) 


III .  Study  Focus 

•  System  Functions  to  be  included 
General 

Function  Operation  _ _  Maintenance  _  Support 

All  Functions _  All  Maint  Functions _  All  Sppt  Fun. 


Subfunctions 


System  Equipments  to  be  included 


Table  1-7  (continued) 


Figures-of-Mer it  (FOM) 

-  Cost 

-  Efficiency 

-  Effectiveness 

-  Congruence  with  Training  Development  Guidelines 

-  Congruence  with  Program  Requirements 

-  Training  Complexity 

-  Training  Capacity 

-  Feasibility 

-  Impact  on  System  Performance 

-  Summary  Evaluation  Score  (Place  Check  Beside  FOM 

Included  in  Score) 


[4 
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Teap  Procedures  to  be  Applied 

Function  Subfunction  Procedures 

_  1.1  _  _ 

_  1.2  _  _ 

1.3  _  _ 

_  1.4  _  _ 

_  1.5  _  _ _ 

_  1.6  _  _ 

__  1.7  _  _ 

_  2.1  _ _ 

_  2.2  _ _ 

2.3  _ _ 

6.1.1 _ _ 

_  6.1.2  _  _ 

6.1.3  _ _ 

_  6.1.4  _  _ 

_  6.2.1  _  _ 

6.2.2  _  _ 

_  6.2.3  _  _ 

6.2.4  _  _ 

_  6.2.5  _  _ 

_  6.2.6  _  _ 

_  6.2.7  _  _ 

_  6.3.1  _  _ 

6.3.2 
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Table  1-7  (continued) 

Evaluative  Technology  Procedures  to  be  Applied 


Function  Subfunction  Procedures 


(4)  What  data  must  be  collected  to  support  the  study? 

An  overview  of  the  procedures  for  developing  the  study  plan 
is  shown  in  Figure  1-11.  The  analyst  must  first  determine 
the  total  amount  of  resources  available  to  conduct  the 
analysis  (see  Table  1-7).  The  resource  and  cost  elements 
which  are  considered  include  labor,  computer  resources,  and 
total  available  budget.  The  available  resources  place 
constraints  on  what  can  be  accomplished  in  any  particular 
ETES  application. 

For  some  functions,  ETES  provides  alternative  procedures  for 
accomplishing  several  key  training  estimation  functions. 
Resource  constraints  may  place  limits  on  which  of  these 
procedures  the  analyst  may  employ.  Generally,  the  less  data 
intensive  procedures  are  the  least  costly.  The  analyst  must 
select  the  procedure  which  fits  his/her  "pocketbook" . 

After  the  study  resources  have  been  determined  the  scope  of 
the  analysis,  or  more  specifically,  the  specific  ETES 
functions  and  procedures  to  be  applied  during  the  proposed 
application  must  be  determined.  ETES  is  an  iterative  set  of 
procedures  which  can  be  applied  repeatedly  during  the  acqui¬ 
sition  process.  However,  it  may  not  be  feasible  to  apply 
all  of  the  ETES  procedures  during  each  application.  Indeed, 
there  is  a  logical  sequence  for  applying  the  ETES  proced¬ 
ures,  (see  Table  1-8).  Thus,  as  can  be  seen  in  Table  1-8,  a 
logical  starting  point  for  the  application  of  the  ETES  would 
be  to  identify  system  functional  requirements.  If  additional 
resources  are  available,  the  user  would  proceed  to  identify 
the  system  hardware/software  design,  and  system  task 
requirements  and  then  to  estimate  training  programs  and 
paths,  etc. 
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Figure  Ml.  Constructing  a  Study  Plan. 
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Table  1-8.  Logical  Sequence  for  Applying 
ETES  Functions/Procedures 


Function 

1.0  Conduct  Functional  Requirements  Analysis 

2.0  Generate  Tasks 

3.0  Estimate  Training  Program 

4.0  Estimate  Training  Resource 

5.0  Determine  Training  Cost 

6.0  Estimate  Training  Efficiency/Effectiveness 
7.0  Evaluate  Training  Program 


Sequence  Grouping 


1 

2 

3 

4 

4 

5 

6 


8.0  Develop  Input  to  Acquisition  Process 


7 


Following  the  identification  of  the  study  scope  the  study 
focus  must  be  determined.  Study  focus  refers  to  the  level 
of  detail  to  which  the  individual  ETES  procedures  will  be 
applied.  Specification  of  the  focus  requires  identifying 
(a)  system  functions  to  be  examined,  (b)  system  equipments 
to  be  examined,  (c)  MOS/duty  positions  to  be  examined,  (d) 
types  of  training  to  be  examined,  and  (e)  f igures-of-merit 
to  be  used  in  evaluating  the  training  programs.  Eight 
f igures-of-merit  are  used  in  ETES:  (1)  cost,  (2)  training 
efficiency,  (3)  training  effectiveness,  (4)  congruence  with 
training  development  guidelines,  (5)  congruence  with  program 
requirements,  (6)  training  complexity,  (7)  training 
capacity,  and  (8)  a  summary  evaluating  score  combining  the 
cost-effectiveness,  and  complexity  measures. 

The  last  step  in  the  development  of  the  study  plan  is  the 
construction  of  the  data  source  index.  The  data  source 
index  lists  the  major  categories  of  data  elements  which  must 
be  collected  during  the  study  and  the  data  sources  from 
which  these  elements  can  be  obtained  (see  Section  1.7.1  for 
a  more  detailed  description  of  the  data  source  index  and  its 
role  in  overall  ETES  data  management).  A  generic  ETES  data 
source  index  encompassing  all  ETES  functions  is  provided  in 
Appendix  E. 

An  overview  of  the  procedures  for  constructing  a  ETES  data 
source  index  is  presented  in  Figure  3-12.  The  user  begins 
the  construction  of  the  data  source  index  by  examining  the 
data  categories  listed  in  the  generic  data  source  index  in 
Appendix  E.  Using  the  information  from  the  ETES  study  plan, 
the  user  then  selects  the  generic  data  elements  needed  to 
support  the  particular  ETES  functions  to  be  applied.  The 
user  subsequently  identifies  lower  level  data  elements  by 
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Figure  1-12.  Procedures  for  Developing  Data  Source  Index. 
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examining  the  system  elements,  functions,  equipments  and 
MOS/duty  positions  to  which  the  procedures  will  be  applied. 
These  system  elements  are  described  in  the  description  of 
the  study  focus  of  the  EXES  study  plan. 

As  soon  as  the  data  source  index  is  constructed,  the  user 
should  begin  to  identify  interface  requirements  for  all 
manual  and  automated  data  bases  which  must  be  accessed 
during  the  ETES  application  (see  Figure  1-12).  Users  should 
contact  the  relevant  POCs  for  each  data  source  to  obtain  up- 
to-date  versions  of  their  standard  operating  procedures 
(SOPs)  for  data  requests.  (Since  SOPs  change  frequently, 
users  should  make  sure  they  have  up-to-date  SOPs  from  the 
cognizant  organizations. ) 

1.8  FORMAT  USED  TO  DESCRIBE  ETES  PROCEDURES 

The  remaining  sections  of  this  guide  (Sections  1-8)  provided 
detailed  descriptions  of  the  ETES  procedures.  Table  1-9 
lists  the  procedures  which  are  presented  in  each  section. 
As  one  would  expect  from  a  training-oriented  system,  ETES 
procedures  for  identifying  training-related  produces 
(Procedures  2-8)  are  described  in  more  detailed  than  the 
procedures  for  identifying  functional  requirements  and  other 
related  information  (Procedure  1). 
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Table  1-9.  Organization  of  Remaining  Sections 


1.0  Conduct  Functional  Requirements  Analysis 

1.1  Identify  Required  System  Functions 

1.2  Identify  System  Performance  Measures  and  Goals 

1.3  Allocate  Required  System  Functions  and  Identify 

Lower  Level  Functions 

1.4  Establish  Baseline  Comparison  System 

1.5  Identify  Required  System  Improvements/New  Technologies 

1.6  Format  New  System  Description 

2.0  Generate  Tasks 

2.1  Identify,  Collect  and  Format  Data 

2.2  Develop  BCS  Tasks 

2.3  Assign  BCS  Tasks  to  MOS,  Duty  Position  and  Skill  Level 

2.4  Develop  New  System  Task  List 

2.5  Assign  New  System  Tasks  to  MOS,  Duty  Position  and 

Skill  Level 

3.0  Estimate  Training  Program 

3.1  Select  Tasks  for  Training 

3.2  Assign  Tasks  to  Training  Settings 

3.3  Identify  Skills  and  Knowledges 

3.4  Develop  Target  Population  Description 

3.5  Sequence  Tasks/Skills  for  Training 

3.6  Construct  Quasi-Program  of  Instructions 

3.7  Assign  Tasks/Skills  to  Media 

3.8  Construct  Training  Paths 

4.0  Estimate  Training  Resource 

4.1  Develop  Course  Operating  and  Support  Plan 

4.2  Determine  Number  of  Students  to  be  Trained 

4.3  Determine  Instructor  Requirements 

4.4  Determine  Facilities  Requirements 

4.5  Determine  Training  Device  and  Training  Equipment 

Requirements 

4.6  Determine  Requirements  for  Other  Training  Resources 

5.0  Determine  Training  Cost 

5.1  Estimate  Individual  Course  Costs 

5.2  Aggregate  Course  Costs 

6.0  Estimate  Training  Efficiency /Effectiveness 

6.1  Estimate  Training  Efficiency 

6.2  Estimate  Training  Effectiveness 
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Table  1-9  (Continued) 


Evaluate  Training  Program 

7.1  Ident ify /Construct  Figures  of  Merit 

7.2  Identify  Likely  Problem  Areas 

7.3  Identify  Likely  Causes 

7.4  Identify  Alternatives 

7.5  Evaluate  Alternatives 

7.6  Assess  Impact  of  System  Changes 

Develop  Input  to  Acquisition  Process 

8.1  Develop/Monitor  Training  Development  Schedule 

8.2  Develop  Inputs  to  Acquisition  Processes  and 

Documents 


SECTION  1.0 


CONDUCT  FUNCTIONAL  REQUIREMENTS  ANALYSIS 


OVERVIEW 

During  this  procedure,  system  functional  requirements  are 
determined  and  functions  are  allocated  to  man,  machine,  and 
software.  To  assist  in  the  identification  of  manpower, 
personnel,  and  training  requirements,  design  concepts  for  a 
Baseline  Comparison  System  (BCS)  and  New  system  are 
established. 

The  results  of  the  functional  analysis  are  entered  into  the 
System  Description  Technology  (SDT).  The  SDT  serves  as  the 
repository  for  information  and  data  developed  during 
functional  analysis  and  insures  that  subsequent  analyses  of 
MPT  requirements  will  be  based  on  a  common  set  of  data. 

PROCEDURE 

This  procedure  consists  of  six  lower  level  procedures. 
During  the  first  procedure,  high  level  system  functional 
requirements  are  identified.  In  the  second  procedure, 
performance  measures  and  goals  are  constructed  for  these 
functions.  In  the  third  procedure,  lower  level  system 
functions  are  identified.  In  the  fourth  procedure,  the 
Baseline  Comparison  System  (BCS)  is  constructed.  In  the 
fifth  procedure)  required  system  improvements  and  new 
technologies  are  identified.  In  the  sixth  procedure,  these 
new  technologies  are  integrated  into  a  systematic  design 
concept  for  the  New  system. 


The  last  two  procedures  are  not  necessary  if  a  New  system 
design  concept  is  available  from  the  Program  Office. 
However,  relevant  information  on  the  New  system  should  be 
entered  into  the  SDT  using  the  guidelines  contained  in 
Procedures  1.5.3  and  1.6.3. 

Potential  data  sources  for  functional  requirements  analysis 
procedures  are  listed  in  Appendix  E. 

1.1  IDENTIFY  REQUIRED  SYSTEM  FUNCTIONS  AND  DETERMINE  MPT 
STUDY  REQUIREMENTS 

OVERVIEW 

The  first  step  in  conducting  system  functional  analysis  is 
to  identify  system  functional  requirements.  This  step  is 
initiated  by  obtaining  and  reviewing  any  available 
functional  requirements  information  on  the  New  system  such 
as  that  contained  in  the  Justification  for  Major  System  New 
Start  { JMSNS )  and  threat  studies  as  well  as  similar 
information  on  the  predecessor  or  other  comparable  systems. 
This  information  is  used  to  identify  and  describe  system 
missions  and  to  generate  high  level  functions  and 
subf unct ions . 

PROCEDURE 

An  overview  of  procedure  1.1  is  provided  in  Figure  1-1. 

1.1.1  Identify  Acquisition  Characteristics 

During  the  earliest  phases  of  the  acquisition  process, 
information  on  system  functional  requirements 


can  be 


PREDECESSOR/ 
COMPARABLE 
SYSTEMS}  I 
information/ 


obtained  from  the  Directorate  of  Combat  Developments  (DCD) 
at  the  TRADOC  School  associated  with  the  developing 
system.  Later,  after  a  Program  Manager  (PM)  has  been 

appointed,  this  information  can  be  obtained  from  the  Program 
Office.  The  best  source  of  information  for  functional 

requirements  information  during  the  early  phases  of  a  new 
acquisition  is  the  Justification  of  a  Major  System  New  Start 
( JMSNS ) ,  formerly  called  a  Mission  Element  Needs  Statement 
(MENS).  The  purpose  of  the  JMSNS  is  to  identify  a  new 

threat  and  the  requirement  for  a  new  weapon  system.  The 
JMSNS  also  explains  the  system's  general  operational  and 
support  envelope  (as  required  in  Office  of  Management  and 
Budget  (OMB)  Circular  A-109,  DoD  Direction  5000.1,  and  DoD 
Instruction  5000.2).  An  outline  of  the  information  elements 
contained  in  the  JMSNS  is  presented  in  Table  1-1. 

Table  1-1.  Outline  of  Information  Elements 
Contained  in  JMSNS 

o  Defense  Guidance  Element 
o  Mission  and  Threat 

o  Alternative  Concepts 

o  Technology  Involved 

o  Funding  Implications 

o  Constraints 

o  Acquisition  Strategy 

The  JMSNS  must  be  supplemented  with  additional  information 
such  as  feasibility  or  technology  base  studies,  operation 
and  support  plans,  mission  area  analysis  results  or  other- 
requirements  documents.  The  availability  of  this  kind  >• 


documentation  depends  on  the  phase  of  the  acquisition 
process  the  system  has  reached.  Examples  of  documents  which 
may  contain  information  on  general  weapon  system  require¬ 
ments  are  provided  in  Table  1-2.  The  documents  listed  in 
Table  1-2  should  be  available  from  (a)  the  Directorate  of 
Combat  Developments  prior  to  the  establishment  of  a  PM  or 
(b)  the  weapon  system  Program  Management  Office  (PMO). 


Table  1-2.  List  of  Program  Documents 
For  a  New  System 


o  Justification  for  Major  System  New  Start  (JMSNS) 
o  Mission  Area  Analysis  (MAA) 

o  Technology  base  studies 

o  Army  "86"  and  "90"  studies 

o  Cost  and  Operational  Effectiveness  Analyses  (COEA) 
o  TRADOC  Scenarios 

o  Concept  Analysis  Agency  Studies 

o  Battlefield  Development  Plan  ( BDP ) 

o  Scenario  Oriented  Recurring  Evaluation  System 
(SCORES) 

o  Threat  studies  by  OACSI/INSCOM 

o  Operational  and  Organizational  Concepts 

o  Analysis  of  Technological  Opportunities  by  TRADOC/ 

DARCOM 

o  Operation  and  Support  Plans 

Following  the  collection  of  available  program  documentation, 
a  list  of  points  of  contact  for  obtaining  additional  system 
and  acquisition  program  information  should  be  developed. 
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Identification  of  points-of-contact  begins  with  the 
collection  of  up-to-date  organizational  charts  for  the  PMO, 
DCD,  and  major  contractors  (if  a  contractor  has  been 
selected).  Points  of  contact  are  then  interviewed  (either 
in  person  or  by  phone)  to  obtain  additional  system  informa¬ 
tion,  program  documentation,  or  further  points  of  contact. 

The  weapon  system  requirements  documentation  will  often  list 
acquisition  objectives  and  constraints  which  are  not  related 
to  the  performance  of  the  system  (e.g.,  cost  and  schedule 
constraints).  These  objectives  and  constraints  may  impact 
MPT  requirements.  The  system  acquisition  schedule  is  of 
particular  concern  in  this  regard.  The  rate  at  which  new 
systems  move  through  the  acquisition  process  and  are  fielded 
directly  impacts  the  requirements  for  manpower,  personnel, 
and  training  resources.  MPT  requirements  can  be  developed 
for  either  steady  state  (constant  number  of  systems 
operating  per  year)  or  phased  situations  (accounting  for 
installation/retirement  schedules).  Each  situation  requires 
different  types  of  acquisition  schedule  information.  All 
acquisition  objectives  and  constraints  including  the  acqui¬ 
sition  schedule  should  be  documented  on  an  Acquisition 
Information  Worksheet  (ACQ).  Figure  1-2  contains  a  sample 
ACQ  worksheet. 

1.1.2  Identify/Describe  System  Missions 

This  step  involves  developing  a  mission  prof ile/operational 
mode  summary  which  contains  a  list  of  the  New  System 
mission(s)  and  the  corresponding  operational  requirements. 
The  mission  profile  consists  of  two  parts;  (1)  a  narrative 
description  of  potential  New  system  missions  and  (2) 
quantitative  data  describing  the  operational  requirements  of 
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those  missions.  A  worksheet  for  documenting  mission  profile 
data  is  presented  in  Figure  1-3. 

Mission  profile  source  documents  may  be  obtained  from  the 
Directorate  of  Combat  Developments  ( DCD )  associated  with  the 
New  system  or  the  system  program  office.  However,  if  this 
information  is  not  available,  the  mission  profile  must  be 
estimated  via  comparability  analyses  utilizing  mission 
information  for  the  predecessor  or  other  comparable  systems 
(procedures  for  identifying  comparable  systems  are  described 
in  the  Introduction  and  in  Procedure  1.4). 

Program  documentation  is  used  to  identify  and  match  New 
system  missions  (columns  c  and  d)  with  predecessor/ 
comparable  system  missions  (columns  a  and  b)  .  If  informa¬ 
tion  on  the  New  system  functions  is  already  available, 
columns  (a)  and  (b)  may  be  left  blank.  Columns  (e)  and  (f) 
describing  the  system  functions  should  be  filled  in  when 
Procedure  1.1.3  (Determine  High  Level  Functions)  is 
completed. 

Quantitative  data  describing  each  mission  function  should  be 
entered  into  columns  (g)  thru  (k).  These  columns  contain 
information  on: 


percent  operating  time  (column  g), 
annual  number  of  missions  (column  h), 
annual  operating  days  (column  i), 
mean  duration  (column  j),  and 

operating  requirements  for  new  system,  in  terms  of 
key  performance  measures  (e.g.,  miles  driven  per 
year)  (column  k) . 


System 


osws 


Figure  1-3  Mission 
Worksheet 


Procedure  1.1 


Pred«ctssor/Comparabl«  System  j 

New  System 

Mission 

Name 

(a) 

Mission 

Character  istics 

(b) 

Mission 

Name* 

(0 

Mission  Charactaristics 

Table  1-3 

Table  1-3 

Provide  Fire 

Short/Medium  Range 

provide 

Dong/Short  Range  fice  for  purposes  of: 

Support 

fire  for  purposes  of 

Fire  Support 

(1)  target  engagements 

(1)  target  engagements 

(2)  counter  fire 

(3)  counterfire 

(3)  air  defense  suppression 

(4)  intordiction/deep  fires 

Survive  in  a 

Periodic  movement 

Survive  in  a 

Rapid  and  frequent  movement  to  confuse  enemy 

Battlefield 

Secondary  armament 

Battlefield 

counterfire  targeting  ability  and  mask  fore 

Environment 

Environment 

Defend  against  anticipated  NBC  environment 

secondary  armament 

Sustain  Weapon 

Field  resupply  capability 

Sustain  Weapon 

Provide  for  continuous  resupply  of  ammuntttc 

Operations 

Operations 

Fill  maintenance  capability 

Field  maintenance  capability 

Tactical  Positioning 

Rapid  and  frequent  movement  over  relatively 

distances  while  maintaining  continuous  flrt 

operations 

Major  System 
Functions  Required 
by  Mission 


Function  (f) 

Percent  Operating  Time  (g) 

Number  of  annual 
Missions  (h) 

Annual  Operating 
Days  (i) 

Mean  Duration 
(l) 

Operating  Req  (k) 


If  quantitative  mission  data  is  not  available  for  the  New 
system,  this  information  should  be  estimated  by  using  data 
from  the  predecessor  or  other  comparable  systems. 

1.1.3  Determine  High-Level  Functions 

In  this  step,  major  system  functions  are  identified  and  a 
function  hierarchy  is  developed.  Major  system  functions 
should  be  listed  in  the  program  documentation  identified  in 
Procedure  i.1.2.  These  major  functions  form  the  framework 
for  the  system  functional  hierarchy.  These  major  functions 
are  then  developed  into  a  hierarchy,  and  listed  in  the  first 
two  columns  of  the  System  Performance  Specification  work¬ 
sheet  ( PRF ) ,  displayed  in  Figure  1-4  (the  remaining  columns 
in  this  worksheet  are  completed  in  Procedures  1.2  and 
1.3).  The  entire  system  functional  hierarchy  goes  beyond 
identification  of  the  major  functions;  it  is  a  hierarchical 
listing  of  the  functions  which  the  new  system  must  perform 
from  the  most  general  functional  level  (e.g.,  fire  weapon  at 
target)  down  to  the  level  where  specific  subsystem  functions 
(e.g.,  locate  targets)  are  identified.  Identification  of 
functions  below  the  subsystem  level  is  accomplished  in 
Procedure  1.3.  High  level  functions  should  describe  what 
the  system  must  do  not  how  it  will  do  it.  Examples  of  high 
level  functions  are  presented  in  Figure  1-4. 

Descriptions  of  system  functions  are  likely  to  be  available 
in  the  program  documentation.  However,  program  documen¬ 
tation  and  information  are  often  incomplete  and  do  not 
provide  a  systematic  listing  of  system  functions.  Often,  for 
instance,  the  program  information  focuses  on  those  functions 
which  are  associated  with  potential  improvements  of  the 
existing  system.  This  treatment  ignores  existing  functions 
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which  still  must  be  performed  by  the  system  but  not  neces¬ 
sarily  in  an  improved  manner.  Therefore,  the  program 
information  may  have  to  be  supplemented  in  order  to  develop 
a  complete  set  of  hierarchical  system  functions.  To  accom¬ 
plish  this,  the  user  may  also  have  to  obtain  functional 
information  on  the  predecessor  system  and/or  other  existing 
systems  which  perform  functions  similar  to  those  required  of 
the  New  system.  More  details  on  the  identification  of 
predecessor  and  comparable  system  data  and  the  identifi¬ 
cation  of  high  level  functions  are  provided  in  the 
paragraphs  below. 

O  IDENTIFICATION  OF  PREDECESSOR  AND  OTHE  COMPARABLE 
EXISTING  SYSTEMS 

The  Predecessor  system  (that  is,  the  system  which  the  new 
system  is  designed  to  replace)  should  be  identified  in  the 
requirements  documents  identified  during  Procedure  1.1. 


Comparable  existing  systems  are  identified  in  a  two-step 
process.  First,  the  functional  area  associated  with  the  New 
system  is  identified.  A  list  of  functional  areas  and  the 
major  systems  within  these  functional  areas  is  provided  in 
Table  1-3.  Second,  the  existing  system  within  the  func¬ 
tional  area  which  most  closely  matches  the  requirements  of 
the  New  system  is  identified.  If  the  user  is  not  familiar 
with  the  systems  in  the  functional  area,  he/she  should 
contact  subject  experts  from  the  school  within  the  desig¬ 
nated  functional  areas  to  obtain  assistance  in  identifying 
comparable  existing  systems. 


TABLE  1-3.  SYSTEMS  AND  FUNCTIONAL  AREA  PROPONENTS* 


PROPONENT 

US  Army  Air 
Defense  School 


US  Army  Armor 
School 


SYSTEMS  AND  AREAS 


System  Peculiar  Equipment; 

HAWK  Missile  (Operator /Organizational 
maintenance) 

HERCULES  Missile  (Operator /Organizat ional 
maintenance) 

CHAPARRAL  Missile  (Operator /Or ganizational 
maintenance) 

REDEYE  Missile  (Operator /Organizational 
maintenance) 

ROLAND  Missile  (Operator /Organizational 
maintenance) 

VULCAN  Gun  (Operator /Organizational 
maintenance) 

M42  (40mm  Self-Propelled) 

Command  and  Control  Radar  Systems 
NA/TSQ-73  (Organizational,  DS/GS 
maintenance) 

AN/TSQ-38  (Organizational,  DS/GS 
maintenance) 

FAAR  (Organizational  maintenance) 

PATRIOT  (Operator /Organizational 
maintenance) 

DIVA1)  Gun  (Operator/Organizational 
maintenance) 

Functional  Areas: 

Air  Defense  System;  Employment  and  Operations 

System  t'^cul^ar  Equipment: 

Techniques  of  Firing: 

Submachinegun  (M3A1) 

Coaxial  Machinegun 
M85-50  Machinegun 
Armored  Vehicle  Operations 
XM1  Tank 

M551  (Armored  Reconnaissance  Vehicle) 

M557  (Command  Post  Vehicle) 

M578  (Light  Recovery  Vehicle) 

M88  (Tank  Recovery  Vehicle) 

Calvary  Fighting  Vehicle  (CFV) 


*  Taken  from  TRADOC  Cir  351-1.  Common  Job  and  Task  Management. 


TABLE  1-3.  SYSTEMS  AND  FUNCTIONAL  AREA  PROPONENTS  (continued) 

PROPONENT  SYSTEMS  AND  AREAS 

US  Army  Armor 
School  (Cont) 


Functional  Areas: 
Combat  Reconnaissance 


-  ) 


m 

i 
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US  Army  Chaplain 
Center  and  School 


US  Army  Engineer 
School 


US  Army  Field 
Artillery  School 


Functional  Areas: 
Religious  Activities 
Pastoral  Counseling 


System  Peculiar  Equipment: 

Prime  Power  Generation  Equipment 
Combat  Engineer  Vehicle  (CEV) 

Armored  Vehicle  Launched  Bridge  (AVLB) 
Engineer  Construction  Equipment 
Engineer  Support  Equipment 
Tactical  Utility /Precise  Power  Electric 
Generation  Equipment  (Operator /Crew, 
DS/GS  maintenance) 
Topographic/Cartographic  Equipment 
Welding  Equipment  -  ARC  (DS/GS 
maintenance) 

Night  Vision  Material  (other  than 

Common  Thermal)  (DS/GS  maintenance) 


Functional  Areas: 

Mine /countermine  Operations 

Military  Bridging 

Map  Reading 

Camouflage  Operations 

Demolitions 

Field  Fortifications 

Route  Reconnaissance 

Obstacles 

Fire  Protection/Fire  Safety 
Refrigeration,  Air  Conditioning,  Heating 
Environmental  Protection 


System  Peculiar  Equipment: 

Cannons  (Operator  maintenance) 

LANCE  Missile  (Organizational 
maintenance) 

PERSHING  Missile  (Organizational 
maintenance) 

Field  Artillery  Target  Acquisition 
systems 
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TABLE  1-3.  SYSTEMS  AND  FUNCTIONAL  AREA  PROPONENTS  (continued) 


PROPONENT 


SYSTEMS  AND  AREAS 


US  Army  Field 
Artillery  School 
(Cont) 


Functional  Areas: 
Observed  Fire 


US  Army  Infantry 
School 


System  Peculiar  Equipment: 

Rifles 

Machine guns 

Grenade  Launcher  M203 

Rocket  Launcher  M20A1,  M20B1 

HEAT  Rockets 

Flame  Weapons 

Weapons  Oriented  Night  Vision  Devices 

PVS-2 

PVS-5 

TVS-2 

TVS-4 

Anti-Tank  Weapons 

TOW  (Operator  maintenance) 

DRAGON  (Operator  maintenance) 

LAW 

106  RCLR 
90  RCLR 
Hand  Grenades 

Infantry  Fighting  Vehicle  (IFV) 

M113A1  APC 
M114  APC 

Improved  TOW  Vehicle  (ITV) 

Functional  Areas; 

Operations 

Ranger  Operations 

Physical  Fitness 

Visual  Signals 

Leadership 

Anti-Armor  Warfare 

Jungle  Operations 

Air  Mobile  Operations 

Air  Assault  Operations 

Airborne  Operations 

Land  Navigation 

Security  and  Combat  Patrols 

Early  Warning  Devices 


TABLE  1-3.  SYSTEMS  AND  FUNCTIONAL  AREA  PROPONENTS  (continued) 


PROPONENT 

US  Army  Intelligence 
School  (Devens) 


US  Army  Intelligence 
Center  and  School 
(Huachuca) 


US  Army  Military 
Police  School 


US  Army  Missile 
and  Munitions 
Center  &  School 


SYSTEMS  AND  AREAS 


System  Peculiar  Equipment: 

Electronic  Warfare/Signal  Intelligence 
(EW/SIGINT)  Analysis 
EW/SIGINT  Non-Morse  Interception 
EW/SIGINT  Morse  Interception 
EW/SIGINT  Voice  Interception 
EW/SIGINT  Emitter  Identification/Location 
EW/SIGINT  Noncommunications  Interception 
EW/INTERCEPT  Equipment  Repair 
Signal  Security 


Functional  Areas: 

Imagery  Intelligence 
Interrogation 

Ground  Surveillance  Radar  Operations 

Remote  Sensor  Operations 

Combat  Area  Surveillance  Radar  Repair 

Aerial  Surveillance  Sensor  Maintenance 

Aircraft  Survivability  Equipment  Repair 

Intelligence  Analysis 

Counter-Intelligence 

Aerial  Sensor  Operations 

Safeguarding  Classified  Documents 

(except  storage,  receipt,  and  transfer) 
Intelligence 


Functional  Areas: 

Civil  Disturbances 
Physical  Security 
Rear  Area  Protection 
Crime  Prevention 
Criminal  Investigation 
Techniques  of  Firing: 

.45  Caliber  Pistol 

.38  Caliber  Pistol 
Prisoner  of  War 


System  Peculiar  Equipment: 

LANCE  Missile  (DS/GS  maintenance) 
PERSHING  Missile  (DS/GS  maintenance) 
HAWK  Missile  (DS/GS  maintenance) 
HERCULES  Missile  (DS/GS  maintenance) 


TABLE  1-3.  SYSTEMS  AND  FUNCTIONAL  AREA  PROPONENTS  (continued) 


PROPONENT 

US  Army  Missile 
and  Munitions 
Center  &  School 
(Cont) 


US  Army  Ordnance 
and  Chemical  Center 
and  School 


SYSTEMS  AND  AREAS 


CHAPARRAL  Missile  (DS/GS  maintenance) 
REDEYE  Missile  (DS/GS  maintenance) 

ROLAND  Missile  (DS/GS  maintenance) 

PATRIOT  Missile  (DS/GS  maintenance) 

DIVAD  Gun  (DS/GS  maintenance) 

VULCAN  Gun  (DS/GS  maintenance) 

SHILLELAGH  (DS/GS  maintenance) 

TOW  (DS/GS  maintenance) 

DRAGON  (DS/GS  maintenance) 

Forward  Area  Alerting  Radar  (FAAR) 

(DS/GS  maintenance) 

LCSS 

Functional  Areas: 

Munitions  Handling,  Storage,  Inspection, 
and  Maintenance 

Explosive  Ordnance  Disposal  &  Equipment 
Missile  Test  Measurement  and  Diagnostic 
Equipment 

Aviation  Life  Support  Equipment 

(Pyrotechnics-R&D,  storage,  and  issue) 
Calibration  and  Related  Support  Equipment 


System  Peculiar  Equipment; 

Wheeled  Vehicles  (maintenance) 

Tracked  Vehicles  (maintenance) 

Power  Generating  Equipment  (maintenance) 
Materials  Handling  Equipment  (maintenance) 
Small  Arms  (maintenance) 

Artillery  Maintenance  (Non-Missile) 

Ground  Support  Equipment  (maintenance) 
Cannons  (DS/GS  maintenance) 

Chemical  Equipment  (maintenance) 

Turret  Maintenance 
Recovery  (Less  Aircraft/Watercraft) 
Compressors  (Shop)  (Organizational/DS 
maintenance) 

Laser  -  General  Purpose  (Organizational/ 

DS  maintenance) 

Laundry  Equipment  (Organizational/DS 
maintenance) 

Metal  Working  and  Machining  Equipment 
(Organizational/DS  maintenance) 

Tire  Repair  Equipment  (DS  maintenance) 
Welding  Equipment  -  ARC  (Organizational 
maintenance) 

Welding  Equipment  (Other)  (Organizational/ 
DS  maintenance) 


TABLE  1-3.  SYSTEMS  AND  FUNCTIONAL  AREA  PROPONENTS  (continued) 


PROPONENT 

US  Army  Ordnance 
and  Chemical  Center 
and  School  (Cont) 


US  Army 

Quartermaster  School 


US  Army  Signal 
School 


SYSTEMS  AND  AREAS 


Functional  Areas: 

NBC  Defense 

SMOKE  Equipment  and  Employment 
NBC  Defensive  Materiel 
Chemical  Support  Equipment 

(Organizational/DS  maintenance) 


System  Peculiar  Equipment: 

Supply  (Automated  &  Manual) 

Food  Service 

Functional  Areas; 

Airdrop  Equipment  Supply  and  Rigging 
Exchange,  Textile,  and  Leather  Support 
Fabric  Material  and  Repair 
Subsistence  and  Food  Service  Equipment 
Memorial  Activities 
Laundry,  Clothing  Exchange,  and  Bath 
Operations 

Materials  Handling  Equipment  (Operation) 
Office  Machines  and  Repair 
Organization  Maintenance  and  Storage 
of  Small  Arms 

Packaging,  Packing,  and  Crating 
Petroleum  Supply,  Equipment,  and 
Operations 

Procurement  and  Production 
Property  Disposal 

Shelters  (General  Purpose  -  Soft  Wall) 
Storage 

Supply  and  Supply  Management  Operations 
(Except:  Medical,  Ammunition,  ADP, 
and  Cryptographic) 

The  Army  Maintenance  Management  Systems 
(TAMMS) 

Materiel  Management 


System  Peculiar  Equipment: 
Radio/Cable 

Functional  Areas: 
Audiovisual  Equipment 
Antennas 

Message  Processing 
TMDE 


US  Army  Signal 
School  (Cont) 


Functional  Areas: 

Television  Equipment 
TV  Production 
ADP  Equipment 
ADP  Peripheral  Equipment 
Data  Communication  Equipment 
Avionic  Navigation  and  Flight 
Control  Equipment 
Avionic  Communications  Equipment 
Tactical  FM  Radios 
Manual  Switchboards 
C-E  Maintenance  Administration 
Electronic  Switchboards 
Telephones 
Generators  (Signal) 

COMSEC  Equipment 
Patching  Panels 
Converters 

Central  Office  Equipment 
Multiplexers 
Combat  C-E  Planning 
C-E  Engineering 
Power  Supplies 

System/Technical  Control  Equipment 

Printed  Circuit  Boards 

HF  Radios 

Public  Affairs 

Radio  Set  Control  Groups 

Electronic  Switchboards 

Facsimile  Equipment 

Fixed  Telecommunications  Systems  Planning 

Photographic  Equipment 

Watercraft  Navigational  Aids 

Land  Navigation  Positioning  Equipment 

Ground  Control  Approach  Equipment 

Illustration 

Multichannel  Equipment 

Message  Processing 

Material  Management  (for  Signal  Equipment) 
Radio  Wire  Integration  (RWI) 

Radiotelephone  Procedures 

Radio  Teletypewriter  Equipment  (RATT) 

Signal  Security 

Signal  Records  and  Reports 

Specialized  C-E  Skills 

Strategic  Microwave  Equipment 
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TABLE  1-3.  SYSTEMS  AND  FUNCTIONAL  AREA  PROPONENTS  (continued) 


PROPONENT 


SYSTEMS  AND  AREAS 


US  Army  Signal  Strategic  Satellite  Equipment 

School  (Cont)  Specialized  Signal  Equipment 

Teletypewriter  Equipment 
Tactical  Satellite  Equipment 
Tactical  Microwave  Equipment 
Tabulating  Equipment 

Wire  and  Cable  Communications/Operations 


US  Army 

Transportation 

School 


Functional  Areas: 

Wheeled  Vehicle  Operations  (Except 
Special  Use  Equipment) 

Aviation  Maintenance 

Marine  Operations  and  Maintenance 

Rail  Operations  and  Maintenance 


US  Army  Institute  Functional  Areas: 

of  Military  Assistance  Civil  Affairs 

Survival,  Evasion,  Escape,  and 
Resistance  (SEER) 

Psychological  Operations 
Special  Forces  Operations 
Security  Assistance 
Foreign  Area  Specialty 
Low  Intensity  Conflict  (LIC) 

Civil  Military  Operations 
US  Army  Participation  in  Counter 
Terrorist  Operations 
Defense  Against  Enemy  Psychological 
Operations 


US  Army  Admin  Center  Functional  Areas: 

Administrative  Services 

Records  and  files  management 
Correspondence  management 
Routing  of  official  mail 
Micromat ion 
Word  processing 
Forms  management 
Printing  and  reproduction 
Publications  supply 
Command  publications  and  orders 
Postal  operations 


TABLE  1-3.  SYSTEMS  AND  FUNCTIONAL  AREA  PROPONENTS  (continued) 


PROPONENT 

US  Army  Admin 
Center  (Cont) 


SYSTEMS  AND  AREAS 


Functional  Areas: 

Storage,  transfer,  and  control  of 
classified  documents 
Reports  control 
Unit  mail  operations 
Article  15* 

Summary  courts  Martial* 

Special  courts  Martial* 

Article  32  investigations* 

Claims* 

Court  reporting* 

Finance  services 

Military  pay  and  allowances 
Travel  allowances 
Commercial  accounts 
Civilian  pay 
Disbursing  operations 
JUMPS  Army  management 

Comptroller ship 

Planning,  programming  and  budgeting 

Program  review 

Internal  review  (auditing) 

Accounting  systems  and  procedures 
Management  analysis  of  programs 
Manpower  management 

Computer  Science 
ADP  management 
ADP  systems  analysis 
Computer  programming 
ADP  equipment  operations 

Personnel  Management 

Recruiting  and  retention 
Personnel  management  techniques 
Professional  ethics 
Club  and  restaurant  management 
Morale  and  recreation  services 

Personnel  Services 
SIDPERS 

Casualty  reporting 
OERs 


*Enlisted  tasks  only 


TABLE  1-3.  SYSTEMS  ANT)  FUNCTIONAL  AREA  PROPONENTS  (continued) 


PROPONENT 

US  Army  Admin 
Center  (Cont) 


Academy  of  Health 
Sciences,  US  Army 


US  Army  Aviation 
Center  and  Fort  Rucker 


US  Army  Training 
Board 


The  Judge  Advocate 
General  School 


SYSTEMS  AND  AREAS 
EERs/SEERs 

Personnel  assignments /utilization 

Personnel  classification/reclassification 

MOS  classification /reclassification 

Awards  and  decorations 

Separations 

Promotions /reduct ions 

Personnel  requisitioning/PERDDIMS 

Personal  affairs 

Personnel  records 

VTAADS 

Other 

Equal  opportunity 

Personnel  and  financial  management 
information  systems 


Functional  Areas: 

First  Aid 

Lifesaving  Measures 
Medical  Evacuation 
Emergency  Medical  Care 
Dental  Care  &  Services 
Veterinary  Care  &  Services 
Health  Care  Management,  Patient  Care, 
Rehabilitation 
Clinical  Counseling 


System  Peculiar  Equipment: 

Air  Traffic  Control 

Flight  Operation  Coordination 

Functional  Areas: 

Aviation  Safety 
Aviation  Standardization 


Functional  Areas: 

Management  of  Military  Training  in  Units 


Functional  Areas: 

Military  Justice 

Geneva/Hague  Convention  and  Laws  of  War 


o 


IDENTIFICATION  OF  HIGH-LEVEL  FUNCTIONS 


High-level  functions  must  be  identified  down  to  the 
subsystem  level  (i.e.,  the  level  at  which  further 
identification  of  functions  is  not  possible  without 
allocating  functions  to  people,  hardware,  or  software). 

Sources  for  the  identification  of  system  functions  include 
system  doctrinal  literature,  system  operational  concepts, 
mission  area  analyses,  ARTEPs  and  the  Battlefield  Develop¬ 
ment  Plan  for  the  functional  area  involving  the  New  system. 
The  Battlefield  Development  Plan  contains  the  umbrella 
operational  concept  for  the  functional  area.  A  list  of 
doctrinal  literature  for  each  functional  area  is  provided  in 
Table  1-4. 

Once  a  list  of  potential  high  level  functions  is  identified, 
these  functions  should  be  sorted  and  arranged  in  hier¬ 
archical  order. 

Each  function  should  then  be  assigned  a  number  which 
specifies  its  place  in  the  functional  hierarchy.  For 
example,  the  highest  level  functions  would  be  designated 
1.1,  1.2,  1.3  or  2.1,  2.2,  2.3,  etc.  Lower  level  functions, 
then,  would  be  designated  with  3  digits,  etc. 

When  the  identification  of  high  level  functions  has  been 
completed  the  system  function  numbers  titles  should  be 
entered  into  columns  (a)  and  (b)  of  the  PRF  worksheet  (see 
Figure  1-4). 
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TABLE  1-4 

DOCTRINAL  LITERATURE  PROGRAM  PUBLICATIONS  AND  PROPONENTS 


ACADEMY  OF  HEALTH  SCIENCES 


Publications  Title 


8-10 

Health  Svc  Spt  in  T.O. 

8-15 

Health  Svc  in  CZ 

8-21 

Health  Svc  in  Comm  Zone 

8-25 

Cbt  Medic 

8-26 

Dental  Svc 

8-27 

Vet  Svc 

8-999C 

The  Med  Pit 

8-999D 

The  Div  Med  Rn 

8-999F 

The  Combat  Zone  Hospital 

8-999G 

Ground  Evac  Units 

8-999H 

Air  Evac  Units 

8-9991 

The  Clearing  Co 

8-999J 

C  &  C  Combat  Zone  Health  Svc 

8-999K 

The  MEDSON 

8-999L 

MED  Intel 

8-999M 

Dispensary  Detach 

8-999N 

The  COMMZ  Hospital 

8-9990 

Convalescent  Center 

8-999P 

Preventive  Medicine  Detach 

3-999Q 

C  &  C  COMMZ  Health  Svc  Ops 

8-999R 

Med  Lab 

(J)  8-8 

Med  Spt  in  Jt  Ops 

(J)  8-9 

NATO  Handbook  on  the  Medical 
Aspects  of  NBC  Defensive  Ops 

AIR  DEFENSE 

44-1 

ADA  Employment 

44-1-1 

Air  Defense  Artillery  Ops 

44-1-A 

ADA  Materiel 

44-1-B 

ADA  Materiel  (Nike  Here) 

44-2 

ADA  Employment  (Auto  Wpns) 

44-3 

ADA  Employment,  CV 

44-10 

Roland  AD  Bn  Ops 

44-11 

DIVAD  Gun  ADA  Bn  Ops 

44-15 

Patriot  ADA  Bn  Ops 

44-18 

Stinger  Pit  and  Section  Ops 

44-23 

Redeye  Employment 

44-70 

C&C  System  AN/TSQ-73 

44-72 

C&C  of  SHORAD  Wpns 

44-90 

ADA  Employment,  HAWTC 

44-95 

Nike/Herc  Employment 

Remarks 


TRADOC  Approved 
TRADOC  Approved 


TABLE  1-4 

DOCTRINAL  LITERATURE  PROGRAM  PUBLICATIONS  AND  PROPONENTS  (continued) 

Publications  Title  Remarks 


ARMOR 


FM 

17-47 

Air  Cav  Bde  Combat 

Bde 

FM 

17-50 

Attack  Helo  Ops 

FM 

17-95 

Cavalry 

FM 

71-1 

Tank  &  Mech  Inf  Co 

Team 

FM 

71-2 

Tank  &  Mech  Inf  Bn 

Task  Force 

FM 

90-3 

Desert  Ops 

AVIATION 

FM 

1-2 

Aircraft  Battlefield  Counter¬ 
measures  and  Survivability 

FM 

1-60 

Army  Airspace  Mgt  and  Air 

Traffic  in  a  Combat  Zone 

FM 

90-1 

Employment  of  Army  Aviation 

Units  in  a  High  Threat  Environment 

TC 

1-3 

Army  Aviation  Employment  on  an 
Integrated  Battlefield 

CHAPLAIN 

FM 

16-5 

The  Chaplain 

FM 

16-22 

Conducting  Military 

Funerals  and  Memorials 

CHEMICAL  SCHOOL 

FM 

3-4 

Collective  Protection 

FM 

3-5 

NBC  Decontamination 

FM 

3-8 

NBC  Support  Operations 

FM 

3-9 

Military  Chemistry  &  Biology 
Defense 

FM 

3-10-1 

Employment  of  Chemical  Agents 

FM 

3-10-2 

Chemical  Effects  Data 

FM 

3-10-3 

Field  Behavior  of  Chemical  Agents 

FM 

3-12 

Operational  Aspects  Radiological 
Defense  &  Fallout  Predictions 

FM 

3-15 

Nuclear  Accident  Contamination 
Control 

FM 

3-50 

Deliberate  Smoke  Ops 

FM 

3-87 

NBC  Reconn/Decon  Ops 

FM 

3-100 

NBC  Opns 

TABLE  1-4 


DOCTRINAL 


Publications 


M  7-30 

M  22-100 
M  22-101 
M  71-3 
M  71-100 
>1  71-101 
M  90-2 
M  90-6 
M  90-10 

M  90-11 
M  90-13 
M  90-14 
M  100-1 
M  100-5 
M  100-15 
M  100-16 
M  100-26 
M  (J)  100-27 

M  (J)  100-28 

M  (J)  100-40 

M  (J)  100-42 
M  (J)  100-43 
M  101-5 
M  101-5-1 
M  101-10-1 
M  101-10-2 
M  101-10-3(S) 


M  11-23 
M  11-25 
M  11-26 
M  11-27 
M  11-28 
M  11-29 


LITERATURE  PROGRAM  PUBLICATIONS  AND  PROPONENTS  (continued) 


Title 


Remarks 


COMBINED  ARMS  CENTER 


The  Infantry  Airborne  and  Air 
Assault  But  Ops 
Military  Leadership 
Leadership  Counseling 
Armored  &  Mech  Bde  Ops 
Armored  &  Mech  Div  Ops 
Inf,  AB  and  Air  Assault  Div  Ops 
Tactical  Deception 
Mountain  Ops 

Military  Ops  in  Urbanized 

Terrain 

Winter  Ops 

River  Crossing  Ops 

Rear  Area  Cbt  Ops 

The  Army 

Operations 

CORPS 

EAC 

The  Air/Ground  Ops  System 
USA/AF:  Doctrine  for  Theater 
Airlift  and  Jt  Airborne  Ops 
Doctrine  and  Procedures  for 
Airspace  Control  in  CZ 
Armed  Forces  Doctrine  for  Chem 
Warfare  and  Bio  Wpns  Defense 
USA/AF:  Airspace  Mgt  in  AO 
The  Landing  Force 
SOFM:  Staff  Org  and  Procedure 

Operational  Terms  &  Graphics 
SOFM:  Org,  Tech,  Log  Data 

SOFM:  Org,  Tech,  Log  Data 

SOFM:  Org,  Tech,  Log  Data 


TRAD0C  Approved 
TRAD0C  Approved 

TRAD0C  Approved 


TRAD0C  Approved 
TRAD0C  Approved 
TRAD0C  Approved 
TRAD0C  Approved 


TRAD0C  Approved 

TRADOC  Approved 

TRAD0C  Approved 
TRADOC  Approved 
TRADOC  Approved 


COMMUNICATIONS  COMMAND 


Theater  Army  Commo  Cmd 

Signal  Co  Troposcatter 

Signal  Hq  Ops  Co,  Signal  Spt  Co 

Signal  Cable  Construction  Bn 

Signal  Long  Line  Co 

Sig  Msg  Co,  Sig  Radio  Ops  Co 
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TABLE  1-4 


DOCTRINAL  LITERATURE  PROGRAM  PUBLICATIONS  AND  PROPONENTS  (continued) 


Publications 


FM  45-1 


FM  5-100 
FM  5-101 
FM  5-102 
FM  5-103 
FM  5-104 
FM  5-105 
FM  5-106 
FM  5-107 
FM  5-108 


FM  6-1 
FM  6-20 

FM  6-20-1 
FM  6-20-2 
FM  6-42 
FM  6-42-l(C) 
FM  6-50 
FM  6-60 
FM  6-121 
TC  6-1-2 
TC  6-20-3 

TC  6-20-5 
TC  6-20-7 
TC  6-30-1 
TC  6-45-1 
TC  6-50-1 
TC  6-50-2 


Title 


DEFENSE  INFORMATION  SCHOOL 


Public  Affairs  Ops 


ENGINEER 

CBT  Eng  Ops 
Mobility 
Survivability 
Countermobility 
General  Engineering 
Topographic  Ops 
ADM  Ops 

Engineer  Combat  Missions 
Eng  Special  Ops 


FIELD  ARTILLERY 
TACFIRE  Ops 

Fire  Support  in  Combined 
Arms  Ops 
FA  Cannon  Bn 

DIVARTY,  Corps  FAS,  FA  Bde 

Lance  Bn 

Lance  Bn 

FA  Cannon  Btry 

MLRS 

FA  Target  Acquisition 
Battery  Computer  System 
Fire  Support  Operations  in 
Brigade  Sized  Units 
FASCAM 
FAC /FI ST 
GLLD/Copperhead 
MLRS 

Direct  Spt  Cannon  Bn 
FA  Cannon  Unit  Nuclear  Ops 


Remarks 


TRADOC  Approved 


L-3 
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TABLE  1-4 


DOCTRINAL 


Publications 


FM  31-22 

FM  31-26 
FM  33-1 
FM  (J)  41-5 
FM  41-10 
FM  100-20 


FM  7-7 

FM  7-7-1 

FM  7-8 
FM  7-10 
FM  7-20 

FM  7-999A 
FM  21-75 
FM  31-18 

FM  57-38 
FM  71-1 

FM  71-2 

FM  90-4 
FM  90-5 
FM  90-8 
FM  90-10-1 


FM  34-3 
FM  34-4 
FM  34-12 
FM  34-13 
FM  34-23 


LITERATURE  PROGRAM  PUBLICATIONS  AND  PROPONENTS  (continued) 


Title  Remarks 

INSTITUTE  FOR  MILITARY  ASSISTANCE 


Command,  Control  and  Spt  of  SF 
Ops 

SF  Selected  Ops  Tech 
PSYOPS 

Joint  Manual  for  CA  TRADOC  Approved 

Civil  Affairs  Ops  TRADOC  Approved 

Low  Intensity  Conflict 


INFANTRY 


The  Mechanized  Infantry 
Platoon/Squad 

The  Mechanized  Infantry  Platoon 
and  Squad  (IFV) 

The  Infantry  Platoon/Squad 
The  Rifle  Co 

The  Infantry  Battalion  (Inf, 
Abn,  Air  Asslt,  Rgr) 

Tactical  Employment  of  Mortars 
Combat  Training  of  the  Soldier 
Long  Range  Reconnaissance 
Ranger  Company 
Pathfinder  Operations 
The  Tank  and  Mechanized 
Infantry  Co  Team 
The  Tank  and  Mechanized 
Infantry  Battalion  Task  Force 
Airmobile  Operations 
Jungle  Operations 
Counter-guerrilla  Operations 
An  Infantryman's  Guide  to 
Urban  Combat 


INTELLIGENCE  -  DEVENS 


Intel  Analysis 
CEWI  Commo  &  Data  Clinics 
Collection  and  Jamming  Co 
Pit  Ldr/Tm  Ch  Hdbk 
TAC  Exploitation  Bn 
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DOCTRINAL  LITERATURE  PROGRAM  PUBLICATIONS  AND  PROPONENTS  (continued) 

Publications  Title  Remarks 

INTELLIGENCE  -  DEVENS  (Cont) 


FM 

34-31 

CEWI  Co /Detachment 

FM 

34-32 

CEWI  Spt  for  ADA 

FM 

34-40 

EW  Ops 

FM 

34-41 

Jamming  Hdbk 

FM 

34-51 

SIGINT/EW  Templating 

FM 

34-62 

SIGSEC  Tech 

FM 

90-2A 

Electronic  Deception 

TC 

34-41 

Jamming  Handbook 

FM  34-1 
FM  34-2 
FM  34-10 
FM  34-11 
FM  34-14 
FM  34-20 
FM  34-21 
FM  34-22 
FM  34-30 
FM  34-33 
FM  34-52 
FM  34-53 
FM  34-60 
FM  34-61 
FM  34-65 
FM  34-80 
FM  (J)  34-81 

TC  34-50 


INTELLIGENCE  -  HUACHUCA 

Intel  &  EW  Ops 

Mgt  of  Intel  &  EW  Ops 

CBT  EW  and  Intel  Bn  (Div) 

Ground  Surveillance  Co  CEWI  Bn 

Svc  Spt  Co  CEWI  Bn-Div 

CBT  EW  and  Intel  GP  (Corps) 

Ops  Bn,  CEWI  GP  (Corps) 

Aerial  Exploitation  Bn,  CEWI  GP 
CEWI  Co  (ACR/SEP  Bde) 

CEWI  Spt  to  Arty 
Intel  Interrogation 
Recon  &  Surveillance 
Tactical  Counterintel  Ops 
Counterintel  Special  Ops 
OPSEC  Spt 
BN/Bde  S2  Hdbk 

Weather  Intel  in  Spt  of  Army  TRADOC  Approved 

Tactical  Ops 

Recon/Surveillance  Handbook 


LOGISTICS  CENTER 


FM  29-20 
FM  31-82 
FM  54-6 
FM  54-7 
FM  63-1 
FM  63-2 
FM  63-3 
FM  100-10 


Maintenance  Mgt  in  T.0. 
Base  Development 
Theater  Army  Area  Cmd 
Theater  Army  Logistics 
CSS  Ops-Sep  Bde 
CSS  Ops-Div 
CSS  Ops-Corps 
Combat  Service  Support 


TRADOC  Approved 
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TABLE  1-4 


DOCTRINAL  LITERATURE  PROGRAM  PUBLICATIONS  AND  PROPONENTS  (continued) 

Publications  Title  Remarks 

MISSILE  AND  MUNITIONS  CENTER  AND  SCHOOL 


3-20 

Tech  Escort  Ops 

3-21 

Chem  Accident  Contamination  Ctl 

9-6 

Ammo  Svc  in  the  T.O. 

9-13 

Ammo  Hdbk 

9-15 

Explosive  Ord  Disposal  Unit  Ops 

9-16 

Explosive  Ord  Recon 

9-38 

Conventional  Ammo  Unit  Ops 

9-47(84) 

Special  Ammo  Unit  Ops 

9-59 

Missile  Spt  Unit  Ops 

29-27 

Calibration  Svc  in  the  T.O. 

MILITARY  POLICE 

19-1  (Capstone) 

Operational  Concepts  of  MP  on 
the  Battlefield 

19-4 

MP  Team,  Squad,  Platoon  Cbt  Ops 

19-10 

MP  Operations 

19-15 

Civil  Disturbances 

19-16 

Counterterrorism 

19-20 

Law  Enforcement  Investigation 

19-30 

Physical  Security 

19-40  Enemy  POW,  Civilian,  Internees 

&  Detained  Persons 
FM  19-60  Confinement  and  Correctional 

Treatment  of  Military  Personnel 


NUCLEAR  AND  CHEMICAL  AGENCY 

FM  (J)  101-31-1  SOFM:  Nuke  Wpns  Employ  TRADOC  Approved 

Doctrine  and  Procedures 

FM  (J)  101-31-2  SOFM:  Nuke  Wpns  Employ  TRADOC  Approved 

Effects  Data 

FM  (J)  101-31-3  SOFM:  Nuke  Wpns  Employ  TRADOC  Approved 

Effects  Data 
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DOCTRINAL 


Publications 


29-2 

29-23 

29-24 

29-30-1 

29-35 

29-999 

43-1 

43-1-1 


10-27 

10-60 

(J)  10-63 

10-67 

29-10 

29-19 

29-45 

29-51 

29-52 

29-999B 


12-50 


11-40 

11-44 

11-50 

11-92 

24-1 

24-2 

24-17 


TABLE  1-4 

LITERATURE  PROGRAM  PUBLICATIONS  AND  PROPONENTS  (continued) 


Title  Remarks 

ORDNANCE  CENTER 


Organizational  Maintenance  Ops 
Direct  Spt  Maintenance  Ops 
(nondivisional) 

General  Spt  Maintenance  Ops 

Div  Maintenance  Bn 

Maint  Spt  in  Separate  Bde 

Division  Maintenance  Ops 

Org  Maintenance  Managers  Guide 

Org  Maintenance  Managers  Checklist 


QUARTERMASTER 


General  Supply  in  T.O. 

Subsistence  Supply  and 
Management  in  T.O. 

Handling  of  Deceased  Personnel  TRADOC  Approved 
in  T.O. 

Petrol  Supply  in  T.O. 

Supply  Mgt  in  the  Field  Army 
Repair  Parts  Supply  in  T.O. 

GS  S&S  in  the  Field  Army 
Div  Supply  Ops 

Supply  and  Field  Svc  Ops  in  Sep 
Bde  and  Armored  Cav  Regiments 
Forward  Supply  Co 


SCHOOL  OF  MUSIC 


The  Military  Band 


SIGNAL 

TAC  Audiovisual  Doctrine 
ADA  Signal  Ops  Bn 
CBT  Commo  w/i  the  Div 
CBT  Commo  w/i  the  Corps 
CBT  Commo 
Radio  Freq  Mgt 
TAC  Comm  Ctr  Ops 
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TABLE  1-4 

DOCTRINAL  LITERATURE  PROGRAM  PUBLICATIONS  AND  PROPONENTS  (continued) 

Publications  Title  Remarks 

SIGNAL  (Cont) 


FM 

24-18 

Field  Radio  Tech 

FM 

24-20 

Field  Wire/Field  Cable  Tech 

FM 

24-21 

TAC  Multichannel  Radio  Commo  Tech 

FM 

24-22 

Commo  Electronic  Mgt  System  (CEMS) 

FM 

24-26 

TAC  Auto  Switching  (TTC-38) 

FM 

24-27 

TAC  Auto  Switching  (TTC-39) 

FM 

24-28 

TAC  Auto  Msg  Switching 

FM 

24-29 

TAC  Commo  Center  Facility  &  Ops 
(TSQ-111) 

FM 

24-32 

TAC  Tele  Sys  Planning,  Eng, 
Integration  and  Control 

FM 

24-999 

Cryptologistics  Spt  to  the  Army 
in  the  Field 

FM 

32-30 

EW,  Tactics  of  Defense 

TC 

11-999E 

Camouflage  of  Signal  Sites 

TC 

11-999G 

Airborne  Radio  Relay 

TC 

11-999H 

Unit  Level  Switching 

TC 

24-1 

The  CEO I 

TC 

24-3 

Radio  Wire  Integration 

TC 

24-18 

Commo  in  "The  Come  As  You  Are 

War" 

TC 

32-11 

How  to  get  out  of  a  Jam 

SOLDIER  SUPPORT  CENTER 

FM 

12-3-1 

Company/Bn  Level  P&A  Doctrine 

FM 

12-3-2 

DIV/SEP  Bde  Level  P&A  Doc 

FM 

12-3-3 

CORPS  Level  P&A  Doc 

FM 

12-3-4 

EAC  P&A  Doc 

FM 

12-15 

Wartime  Casualty  Reporting  Sys 

FM 

12-16 

Replacement  Ops 

FM 

14-6 

Comptroller/Finance  Svc  in  T.O. 

FM 

14-7 

Financial  Services 

FM 

22-999A 

Unit  Cohesion 

FM 

22-999B 

Human  Performance  in  Continuous 
Ops 

FM 

26-999A 

Mgt  of  Stress  in  Army  Ops 

23  22  222322  2  22222 


TABLE  1-4 


DOCTRINAL  LITERATURE  PROGRAM  PUBLICATIONS  AND  PROPONENTS  (continued) 


Publications  Title 


Remarks 


TRANSPORTATION  SCHOOL 


(J)  20-12 

Amphib  Embarkation 

29-39 

Marine  Equip  Maintenance  Mgt 

55-1 

Army  Trans  Svc  in  T.0. 

55-2 

Div  Trans  Ops 

55-10 

Army  Movement  Mgt  Units  and 
Procedures 

(J)  55-12 

Movement  of  Army  Units  in  AF 
Aircraft 

(J)  55-19 

Spt  of  Contingency  Forces 

55-20 

Army  Railway  Units  and  Ops 

55-30 

Army  Motor  Transport  Units  and  Ops 

55-40 

Cbt  Svc  Spt  Air  Transport  Ops 

55-41 

Aircraft  Org  Maintenance  Mgt 

55-42 

Army  Aviation  Intermediate 
Maintenance  (AVIM) 

55-45 

Aviation  Spt  Bn 

55-50 

Army  Water  Transport  Ops 

55-60 

Army  Terminal  Ops 

55-999B 

LACV-30  Units  and  Ops 

TRADOC  Approved 


TRADOC  Approved 


1.1.4  Document  Data 


Relevant  data  collected  in  this  procedure  must  be  entered 
into  the  SDT.  This  data  should  be  entered  into  the  SDT 
immediately  after  the  worksheets  contained  in  Figure  1-2, 
1-3,  and  1-4  are  completed.  Table  1-5  describes  the 
sequence  with  which  these  data  should  be  entered  into  the 
SDT.  Additional  guidelines  for  entering  data  into  the  SDT 
is  provided  in  the  SDT  User's  Guide. 

1.2  IDENTIFY  SYSTEM  PERFORMANCE  MEASURES  AND  GOALS 

OVERVIEW 

Once  system  functions  have  been  defined,  system  performance 
measures  and  goals  and  system  organizational,  operational, 
and  support  concepts  must  be  identified.  These  data 
elements  help  to  further  define  the  functional  requirements 
of  the  New  system. 

Many  new  concepts  and  terms  (for  example,  performance 
measure,  performance  goal)  are  introduced  in  this  procedure. 
The  reader  should  examine  the  definitions  of  key  terms  which 
are  listed  in  Table  1-6  before  beginning  this  procedure. 

PROCEDURE 


An  overview  of  the  steps  in  this  procedure  is  presented  in 
Figure  1-5. 


Table  1-6.  Definition  of  Key  Concepts 


o  System  Performance  Measures  -  A  listing  of  the  measures 
which  are  used  to  describe  the  performance  capabilities 
which  must  be  achieved  by  each  system  function.  These 
measures  are  usually  described  in  the  program 

documentation  (e.g.,  speed  might  be  measured  in  miles 
per  hour  (MPH)  or  kilometers  per  hour  (KPH)). 

o  System  Performance  Goals  -  A  description  of  the  goals 
(or  requirements)  which  must  be  achieved  for  each 
performance  measure.  Goals  may  be  stated  in  general 
terms  (e.g.,  increase  by  10%)  or  as  specific 

quantitative  values  (e.g.,  the  exact  goal  is  listed-50 
MPH  or  30.8  KPH)  . 

o  Organizational  Concept  -  A  general  textual  description 
of  the  way  in  which  the  new  system  will  be  manned  (in 
strictly  an  organizational  sense)  and  the  way  in  which 
it  will  fit  into  existing  Army  organizational 
structures. 

o  Operational  Concept  -  A  general  textual  description  of 
the  doctrinal  (how  to  fight)  concepts  that  will  guide 
the  operation  of  the  system  in  both  wartime  and 

peacetime  and  a  description  of  how  the  system  will  fit 
into  more  general  concepts. 

o  Operational  Environment  -  A  textual  description  of  the 
specific  environment,  geographic,  and  threat  conditions 
which  the  projected  system  will  encounter. 

o  Support  Concepts  -  A  general  description  of  how  and 
where  each  individual  Integrated  Logistics  Support 
Concept  should  be  performed  and  which  Army  organiza¬ 
tions  will  perform  these  elements.  As  outlined  in  DoDD 
5000.39,  the  elements  of  ILS  are  (1)  maintenance,  (2) 
manpower  and  personnel,  (3)  supply  support,  (4)  support 
and  test  equipment,  (5)  training  and  training  devices, 
(6)  technical  data,  (7)  computer  resources,  (8) 
packaging,  handling,  storage,  and  transportation,  and 
(9)  facilities. 


1.2.1  Identify  System  Performance  Measures 

You  may  identify  potential  performance  measures  by  examining 
the  program  documentation  (for  example,  the  JSMNS)  collected 
in  Procedure  1.1.1.  Any  measure  describing  the  potential 
performance  capabilities  of  the  system  should  be  treated  as 
a  potential  performance  measure.  To  aid  the  user,  a  list  of 
common  performance  measures  is  presented  in  Appendix  F. 
Examine  the  list  of  system  functions  identified  in  Procedure 
1.1.3  and  identify  the  performance  measures  associated  with 
each  function.  Enter  the  description  of  each  performance 
measure  in  Column  C  of  the  System  Performance  Specifications 
Worksheet  (PRF)  contained  in  Figure  1-6.  (Columns  a  and  b 
of  this  worksheet,  describing  system  functions  should  have 
been  completed  in  Procedure  1.1.3.)  Each  function  may  (1) 
have  more  than  one  performance  measure  associated  with  it  or 
(2)  not  have  any  associated  performance  requirements  and 
therefore  have  no  performance  measures. 

1.2.2  Identify  System  Performance  Goals 

Examine  the  program  documentation  collected  during  procedure 
1.1.1  and  identify  the  quantitative  goals  and  standards 
associated  with  each  of  the  performance  measures  identified 
in  Procedure  1.2.1.  Enter  the  data  in  Column  (d)  of  the  PRF 
Worksheet  listed  in  Figure  1-6.  A  goal  may  be  stated  in 
general  terms  (for  example,  increase  10%)  or  in  terms  of 
specific  quantitative  values  (for  example,  fire  200  rounds 
per  minute).  Each  performance  measure  should  have  at  least 
one  goal. 


Performance  Specification  Worksheet  (PRF) 
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Performance  Specification  Worksheet  (PR F)  (Continued) 
Worksheet  # _ 


Crons  Country «  15-20 


Performance  Specification  Workiheot  (PRF)  (Continued) 


1.2.3  Identify  Operational,  Organizational,  and  Support 
Concepts 


Information  on  the  organizational  concept,  operational 
concept,  and  operational  environment  should  be  available  in 
the  program  documentation  collected  during  Procedure  1.1.1. 
However,  support  concepts  for  many  of  the  elements  of  ILS 
are  likely  to  be  unavailable  during  the  early  phases  of  the 
acquisition  process.  If  the  operational  and  support  concepts 
are  unavailable,  the  user  should  obtain  and  use  the  opera¬ 
tional  and  support  concepts  for  the  predecessor  or  another 
similar  system.  These  concepts  should  then  be  modified  to 
reflect  the  demands  of  the  new  system,  subject  to 
coordination  with,  and  verification  by,  personnel  from  the 
Program  Management  Office  (PMO).  Organizational,  Opera¬ 
tional  and  Support  Concept  data  is  recorded  on  an  Organiza¬ 


tional,  Operational  and  Support  Concept  Worksheet  (OSC), 
Figure  1-7.  As  the  examples  in  Figure  1-7  indicate,  the 
organizational,  operational,  and  support  concepts  consist  of 
a  combination  of  brief  textual  descriptions  and  quantitative 
estimates  of  usage. 

To  further  document  system  functional  requirements  the  user 
may  use  the  worksheet  listed  in  Figure  1-8,  which  briefly 
describes  the  impact  of  the  threat  and  the  operational 
environment  on  system  functions.  This  information  is 
optional.  However,  some  users  find  it  helpful  in  developing 
descriptions  of  task  conditions  and  standards  for  use  in 
Training  and  Soldiers  Manuals.  Information  in  system  threat 
and  operational  environment  impacts  may  be  obtained  from  (1) 
SCORES,  which  contains  general  descriptions  of  threats  and 
scenarios,  (2)  the  threat  analyses  conducted  during  system 
development,  and  (3)  the  program  documentation  collected 
during  Procedure  1.1. 


5 


Depends  upon  storage  capacity 


1.2.4  Document  Data 


Relevant  data  collected  during  Procedure  1.2  must  be  entered 
into  the  SDT.  This  data  should  be  entered  into  the  SDT 
immediately  after  the  worksheets  listed  in  Procedures  1.2.1 
to  1.2.3  have  been  completed.  Table  1-7  describes  the  most 
desirable  order  for  entering  data  into  the  SDT.  Additional 
guidelines  for  entering  data  into  the  SDT  is  provided  in  the 
SDT  User  Guide. 

1.3  ALLOCATE  REQUIRED  SYSTEM  FUNCTIONS  AND  IDENTIFY  LOWER 
LEVEL  FUNCTIONS 

OVERVIEW 

This  procedure  is  designed  to  allocate  system  functions  in 
Procedure  1.2  to  people,  equipment,  and  software  based  on 
the  allocation  of  functions  in  the  predecessor  and  other 
comparable  existing  systems.  This  information  is  used  to 
develop  a  generic  equipment  structure  and  to  identify  lower 
level  functions  and  place  them  in  a  hierarchy. 

PROCEDURE 

An  overview  of  the  steps  comprising  this  procedure  is 
depicted  in  Figure  1-9. 

1.3.1  Allocate  Major  System  Functions/Subfunctions 

In  this  step,  you  must  allocate  the  functions  identified  in 
Procedure  1.2  to  people,  software,  and  equipment  (or  some 
combination  thereof).  High  level  functions  can  be  allocated 
to  people,  hardware,  and  software  by  (a)  examining  how 


RELATED  RELATED  WORKSHEET 

PROCEDURE  DATA  ELEMENT  SDT  ENTITY  WORKSHEET  COLUMN (S)  SEQ  // 


opt  =  optional 


HARDWARE 

FUNCTIONS 


PREDECESSOR 
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FIGURE  1—  9.  Overview  Diagram:  Allocate  Required 
System  Functions  and  Identify  Lower  Level  Functions  (1.3) 


similar  functions  were  allocated  in  the  predecessor  system 
or  other  similar  systems,  and  (b)  updating  these  allocations 
to  account  for  the  performance  goals  associated  with  the  New 
system. 

Once  identified  the  allocations  should  be  entered  in  the 
last  three  columns  of  the  System  Performance  Specification 
Worksheet  (PRF)  shown  in  Figure  1-10. 

1.3.2  Establish  Generic  Equipment  Structure 

A  generic  equipment  structure  describes  the  generic 
equipment  subsystems  (e.g.,  engine)  which  must  be  included 
in  the  New  system.  A  example  of  a  generic  equipment 
structure  is  provided  in  Table  1-8.  The  generic  equipment 
structure  can  be  developed  by  (a)  examining  the  equipments 
in  the  predecessor  system,  (b)  adding  equipments  to  achieve 
the  additional  capabilities  required  by  the  New  system,  and 
(c)  deleting  equipments  which  are  not  needed  in  the  New 
system  (as  indicated  by  the  New  system  functional 
requirements  and  functional  allocation). 

The  generic  equipment  structure  for  the  predecessor  system 
can  be  identified  by  examining  the  maintenance  manuals 
associated  with  the  system.  To  identify  the  generic 
equipments  which  must  be  added  or  deleted  from  the 
predecessor,  present  the  predecessor  equipment  list  and 
function  allocations  along  with  information  on  the  system 
functional  requirements  to  subject  matter  experts  from  the 
schools  associated  with  the  predecessor  system  and  ask  them 
to  make  the  proper  modifications  to  the  predecessor 
equipment  list. 
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Once  the  generic  equipment  structure  has  been  identified  it 
should  be  documented  in  the  Generic  System  Worksheet  (GEN). 
Columns  (a)  and  (b)  of  the  worksheet  should  list  the  number 
and  title  of  the  system  functions  as  described  in  columns 
(a)  and  (b)  of  the  PRF  worksheet.  Column  (c)  of  the  GEN 
worksheet  should  be  used  to  describe  the  generic  equipment 
structure. 

Once  the  generic  equipment  has  been  identified  at  the 
subsystem  level,  lower  level  equipments  must  be  identified 
and  the  resulting  lower  level  functions  must  be  described. 
Information  on  lower  level  functions  should  be  entered  into 
columns  (a)  and  (b)  of  the  GEN  and  PRF  worksheets. 
Information  on  generic  equipment  should  be  entered  into 
column  (c)  of  the  GEN  worksheet. 

As  part  of  the  identification  of  generic  equipment,  you  must 
allocate  functions  to  hardware,  software,  or  humans.  These 
allocations  should  be  documented  in  column  (e)  of  the  PRF 
worksheet  and  columns  (d),  (e),  and  (f)  of  the  GEN  work¬ 
sheet.  In  the  GEN  worksheet  (column  f),  functions  allocated 
to  humans  are  further  broken  down  by  classifying  these 
functions  as  (a)  generic  operator  functions,  (b)  generic 
maintenance  functions,  or  (c)  generic  support  functions,. 

1.3.3  Determine/Describe  Lower  Level  Functions 


Subsystem  level  functions  are  identified  in  the  same  way  as 
high-level  functions.  Lower  level  functions  from  the 
predecessor  or  other  comparable  systems  are  identified  and 
modified  to  reflect  the  New  system  requirements.  Information 
on  the  predecessor  and  other  comparable  equipments  is 
contained  in  the  Generic  System  Worksheet  (Figure  1-11). 


Figure  1-11  Generic  System  Worksheet  (GEN)  (Continued) 


Identification  of  function  sequence  is  accomplished  by 
identifying  function  sequences  from  the  predecessor  or  other 
comparable  existing  systems  and  then  modifying  these 
sequences  to  reflect  the  New  system  operational  concept  and 
functional  requirements.  Function  sequence  information 
describes  (a)  what  functions  must  be  performed  prior  to, 
after,  or  concurrently  with  a  given  function  during  mission 
performance.  The  best  sources  for  this  function  sequence 
information  are  the  doctrinal  manuals  (e.g.,  the  How-to- 
Fight  manuals  and  ARTEPS)  associated  with  the  existing 
systems . 

A  worksheet  which  can  be  used  to  describe  function  sequence 
is  contained  in  Figure  1-12.  Identification  of  function 
sequences  can  be  facilitated  by  drawing  a  function  sequence 
diagram  such  as  that  contained  in  Figure  1-13,  which 
contains  a  high  level  function  sequence  diagram.  A 
preceding  function  is  a  function  which  must  be  performed 
before  the  designated  function.  A  succeeding  function  is  a 
function  which  must  be  performed  after  the  designated 
function.  A  concurrent  function  is  a  function  which  must  be 
performed  at  the  same  time  as  the  designated  function. 

Whenever  comparability  analysis  procedures  are  employed 
during  system  function  requirements  identification,  the 
comparable  existing  systems  and  the  modification  to  these 
systems  should  be  included  in  the  audit  trail.  In  addition, 
all  information  on  functional  requirements  generated  via 
comparability  analysis  should  be  presented  to  the  Program 
Office/Combat  Developments  Directorate  for  review  and 
ultimate  approval. 


1.3.5  Document  Data 


Relevant  data  collected  in  this  procedure  must  be  entered 
into  the  SDT.  This  data  should  be  entered  into  the  SDT 
immediately  after  the  worksheets  contained  in  Figures  1-10, 
1-11,  and  1-13  are  completed.  Table  1-y  describes  the 
sequence  with  which  these  data  should  be  entered  into  the 
SDT.  Additional  guidelines  for  entering  data  into  the  SDT 
are  contained  in  the  SDT  User's  Guide. 

1.4  ESTABLISH  THE  BASELINE  COMPARISON  SYSTEM  (BCS) 

OVERVIEW 

The  BCS  is  a  notional  design  consisting  of  the  existing 
subsystems  which  currently  come  closest  to  meeting  the 
functional  requirements  of  the  New  system.  Since  the  BCS 
only  contains  existing  subsystems,  the  BCS  may  not  meet  all 
of  the  performance  goals  established  for  the  New  system. 

In  order  to  be  selected  for  the  BCS,  a  subsystem  must  meet 
the  following  criteria:  (1)  it  must  be  an  existing 

subsystem  currently  in  the  DoD  inventory,  preferably  in  the 
Army  inventory,  (2)  it  must  be  the  existing  subsystem  which 
most  closely  matches  system  performance  requirements,  and 
(3)  it  must  have  mature  task  data.  The  latter  set  of  data  is 
critical  for  training  requirements  determination. 

PROCEDURE 

An  overview  of  procedure  1.4  is  contained  in  Figure  1-14. 
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FIGURE  1-14.  Overview  Diagram:  Establish 
Baseline  Comparison  System  (BCS). 


1.4.1  Establish  The  Baseline  Comparison  System  (BCS) 


This  step  is  initiated  with  the  construction  of  a  BCS 
equipment  list.  The  BCS  is  comprised  of  comparable  existing 
subsystems  (predecessor  or  other)  from  the  Army  inventory 
which  (a)  best  meets  the  functional  and  performance 
requirements  of  the  New  system  and  (b)  have  mature  task 
data . 

A  worksheet  for  describing  the  BCS  equipments  is  presented 
in  Figure  1-15.  New  system  functions  and  performance  goals 
are  listed  in  columns  (a)  and  (b).  This  information  can  be 
taken  from  the  System  Performance  Specification  Worksheets 
developed  in  procedures  1.1  through  1.3  (See  Figures  1-4,  1- 
6,  and  1-10 ) . 

Second,  generic  equipments  identified  in  Procedure  1.3.2  and 
listed  in  Figure  1-11  (Generic  System  Worksheet)  are  entered 
in  Columns  (c)  and  (d).  These  generic  equipments  designate 
the  type  of  existing  equipment  to  be  considered  for  the 
BCS. 


Once  columns  (a)  thru  (d)  have  been  completed,  the  worksheet 
should  be  presented  to  subject  matter  experts  from  the  func¬ 
tional  areas  associated  with  each  major  subsystem  in  the 
generic  equipment  structure.  These  analysts  should  be  asked 
to  indicate  what  equipments  in  the  predecessor  or  other 
comparable  system  can  (1)  best  meet  the  New  system  perfor¬ 
mance  requirements,  (2)  has  available  mature  reliability, 
availability,  maintainability  (RAM),  workload,  training  and 
other  data,  and  (3)  where  design  information  for  the  new 
system  exists,  has  technology  as  similar  as  possible  to  that 
projected  for  the  New  system.  Additional  guidance  for  the 
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Baseline  Comparison  Equipment  Selection  Worksheet  (BCS) 
Worksheet  # _ 


3403.1  Fuze  Setting  I  3403 


can  be 


ide nt i f ica t ion  of  BCS  equipments 


provided  by 
in  the  functiona] 


examining  existing  equipment  taxonomies  in  the  functional 
areas  associated  with  the  New  system.  Equipment  taxonomies 
list  the  specific  equipments  associated  with  each  category 
in  the  generic  equipment  structure.  Such  taxonomies  are 


likely  to  be  found  in  configuration  or  logistics  management 
directories  associated  with  the  New  system  functional  area. 


Table  1-10  presents  an  example 
taxonomy  which  is  included  in 


Configuration  Directory 


page  from  the  equipment 
the  Aviation  Electronics 


Once  potential  BCS  equipments  have  been  identified,  the 
subject  matter  experts  should  be  asked  to  note  the  degree  to 
which  each  potential  BCS  subsystem  matches  the  performance 
requirements  of  the  New  system  on  a  one  to  seven  scale  (one- 
not  all  and  seven-identical).  One  rating  should  be  obtained 
for  each  potential  BCS  equipment.  These  ratings  should  be 
entered  in  column  (h)  of  the  BCS  worksheet.  Column  (i)  of 
this  worksheet  should  be  used  to  document  the  availability 
of  BCS  data.  Columns  (k)  and  (1)  may  be  used  to  describe 
the  Functional  Group  Code  (FGC)  Work  Breakdown  Code  ( WBC ) , 
Work  Unit  Code  (WUC),  or  Manufacturer's  Part  Number  (MFG's 
P/N)  of  the  selected  BCS  equipments. 


After  the  BCS  equipment  list  has  been  constructed,  detailed 
data  on  the  BCS  equipment  may  be  obtained.  Some  of  this 
information  may  be  obtained  during  the  manpower  requirements 
analysis.  The  BCS  Equipment  Description  Worksheet  contained 
in  Figure  1-16,  should  be  used  to  document  this  data.  The 
major  focus  should  be  on  the  identification  of  equipment 
characteristics  which  are  directly  related  to  task  ident¬ 
ification  and  analysis.  However,  additional  equipment 
characteristics  may  also  be  identified  and  included  in  the 
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SDT.  Figure  1-16  displays  the  additional  equipment 
characteristics  which  may  be  estimated  and  indicates  which 
o£  these  characteristics  are  used  in  task  identification. 
Identification  of  task  elements  can  be  facilitated  by  the 
types  of  controls  and  displays  which  are  associated  with  the 
equipment.  Displays  and  controls  provide  the  focal  point  of 
the  man-machine  interface.  Another  important  set  of  equip¬ 
ment  characteristics  for  task  identification  are  descrip¬ 
tions  of  the  information  inputs  and  outputs  which  are 
associated  with  each  major  equipment  item.  This  input/output 
information  is  critical  for  those  equipments  which  perform 
some  analytical/software  function.  These  equipment  informa¬ 
tion  inputs/outputs  play  a  key  role  in  determining  human 
task  inputs/outputs. 

In  addition,  available  data  on  equipment  reliability  and 
costs  can  be  entered  on  this  worksheet. 

1.4.2  Determine  BCS  Operational  and  Support  Concepts 

After  construction  of  the  BCS  equipment  list,  information  on 
the  operational  and  support  concepts  of  the  BCS  should  be 
obtained.  (Definitions  of  the  terms  operational  and  support 
concepts  are  provided  in  Procedure  1.2}. 

Operational  concepts  are  documented  in  the  TRADOC  Pam  525 
series.  As  described  in  TRADOC  Reg.  11-7,  the  operational 
concept  describes  "the  performance  of  one  or  more  combat, 
combat  support,  or  combat  service  support  functions.  Each 
operational  concept  defines  wha  t  needs  to  be  done;  why  it 
needs  to  be  done;  how  it  is  to  be  done;  where  it  needs  to  be 


done;  and  who  does  it 


(TRADOC  Reg  11-7,  pg .  3). 


6  BCS  Equipment  Description  Worksheet  (EQP) 
Worksheet  _ 


terns  are  optional 


An  overview  of  the  information  contained  in  the  operational 
concept  is  presented  in  Table  1-] 1 , 

Support  concepts  for  BCS  subsystems  should  be  available  in 

(a)  the  program  documentation  for  that  subsystem, 
particularly  logistics  support  analysis  documentation  and 

(b)  maintenance  and  operator  manuals  for  the  equipment. 

BCS  operational  and  support  concepts  should  be  documented  in 
the  OSC  worksheet  listed  in  Figure  1-7. 

1.4.3  Describe  Function  Sequence  and  Information  Flow 

BCS  function  sequence  information  can  be  obtained  from  How¬ 
to-Fight  manuals  and  ARTEPs  associated  with  the  BCS 
equipment.  Function  sequence  information  should  be  entered 
into  the  worksheet  listed  in  Figure  1-17. 

Information  flow  data  describing  the  inputs  and  outputs 
associated  with  each  BCS  equipment  may  also  be  identified 
(this  information  is  particularly  valuable  in  identifying 
task  requirements  for  automated  systems).  Information  flow 
data  should  be  available  in  the  operator  and/or  maintenance 
manuals  associated  with  the  BCS  equipments.  Once  identified, 
the  BCS  information  flow  data  should  be  entered  into  columns 
(j)  and  (k)  of  the  EQP  worksheet  listed  in  Figure  1-16. 

1.4.4  Document  Data 

Relevant  data  collected  in  this  procedure  must  be  entered 
into  the  SDT.  This  data  should  be  entered  as  the  worksheets 
contained  in  Figures  1-15,  1-16,  and  1-17  are  completed. 
Table  1-12  describes  the  sequence  with  which  these  data 


TABLE  1-11.  FORMAT  OF  THE  OPERATIONAL  CONCEPT 
AND  CONCEPT  STATEMENT* 


Edition  (ie.,  1st  Draft 
2nd  Draft,  etc.) 


Date 


1 .  Purpose : 

-  WHY  is  the  concept  needed? 

-  WHAT  needs  to  be  accomplished? 

-  Are  there  influences  of  geography,  climate,  threat,  or  technology? 
If  there  are,  state  them  in  general  terms.  If  necessary,  give  a 
detailed  explanation  in  an  annex. 

-  If  the  concept  is  one  of  a  set,  common  environmental  parameters  may 
be  stated  in  a  preface  to  the  set.  Individual  concepts  or  concept 
statements  need  only  reflect  that  they  are  a  part  of  the  set. 

2 .  Limitations: 

-  State  any  limiting  assumptions  of  facts. 

•  For  example,  a  concept  may  not  apply  to  joint  or  combined 
operations,  or, 

•  A  concept  may  be  applicable  only  to  the  Mideast  or  Centra] 

Europe. 

3 .  The  operational  concept: 

-  HOW  is  the  task  to  be  done? 

»  How  the  operation  will  be  performed  and  how  it  relates  to 
other  operations  performed  Simula  eously  or  in  support. 

-  WHERE  is  it  to  be  done  on  the  battlefield? 

•  Covering  force  area,  the  main  battle  area,  etc. 

-  WHEN  is  it  to  be  done  in  relation  to  other  battlefield  activities? 

-  WHO  does  it? 

•  What  echelon  of  command  performs  this  task? 

4.  ANNEXES  (As  appropriate) 

-  Glossary  of  terms/acronyms 

-  Bibliography 


‘Taken  from  TRADOC  Reg  11-7. 
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should  be  entered  into  the  SDT.  Additional  guidelines  for 
entering  data  into  the  SDT  are  contained  in  the  SDT  User's 
Guide . 

1.5  IDENTIFY  REQUIRED  SYSTEM  IMPROVEMENTS/NEW  TECHNOLOGIES 

OVERVIEW 

The  purpose  of  this  procedure  is  to  identify  system  improve¬ 
ments/new  technologies  required  for  the  New  system.  The  New 
system  is  defined  as  the  best  estimate  of  New  system  design, 
incorporating  modified  or  improved  design  features, 
technological  advances,  new  operating  and  support  concepts, 
and  changes  to  other  system  elements.  As  with  the  BCS,  the 
New  system  must  fulfill  all  functional  requirements.  Unlike 
the  BCS,  however,  the  New  system  is  expected  to  meet  all 
system  performance  criteria.  The  objective  of  this 

procedure  is  to  identify  and  describe  functional  areas  where 
the  BCS  system  fails  to  meet  required  performance  goals  or 
standards . 

PROCEDURE 

An  overview  of  the  steps  comprising  this  procedure  is 
presented  in  Figure  1-18. 

1.5.1  Identify  Required  System  Improvements 

This  procedure  begins  with  the  comparison  of  the  functional 
requirements  for  the  New  system  and  its  associated 
performance  requirements,  to  the  capabilities  of  each  of  the 
BCS  subsystems.  This  comparison  was  initiated  during 
Procedure  1.4  (See  Figure  1-14).  In  this  procedure,  a  more 


FIGURE  1-18.  Overview  Diagram:  Identify  Required 
System  Improvements/New  Technologies  (  1.5). 


detailed  comparison  is  conducted  to  identify  areas  where  the 
BCS  does  not  meet  the  performance  standards  required  by  the 
New  system.  If  a  new  system  description  and/or  equipment 
list  is  available  from  the  contractor,  it  would  be  utilized 
to  identify  performance  improvements  (over  the  BCS)  which 
are  part  of  the  contractor's  design.  Some  of  these 
improvements  may  exceed  stated  system  performance  goals. 
These  additional  performance  improvements  would  also  be 
documented.  The  results  of  this  analysis  are  documented  on 
a  Baseline  Comparison  Performance  Shortfall  Worksheet  (PSH), 
as  shown  in  Figure  1-19. 

First,  the  BCS  equipments  are  entered  into  columns  (a),  (b), 
and  (c).  Second,  the  New  system  performance  requirements 
are  entered  in  column  (d).  Third,  performance  gaps;  that 
is,  areas  where  the  BCS  does  not  meet  the  performance 
requirements  of  the  New  system,  are  identified  and  entered 
into  column  ( e )  . 

The  same  subject  matter  experts  who  assisted  in  the 
identification  of  BCS  subsystems  should  be  used  to  identify 
the  BCS  performance  shortfalls. 

1.5.2  Identify  New  Technologies 


The  purpose  of  this  procedure  is  to  identify  new  tech¬ 

nologies  which  can  satisfy  each  of  the  required  performance 
improvements  identified  by  the  previous  procedure.  The 
estimate  of  the  New  system  design  is  made  by  incorporating 
desired  and  feasible  technological  improvements.  If  a  New 

system  equipment  description  is  available  from  the 
contractor,  the  identification  of  new  technologies  which 

could  meet  each  of  the  required  design  improvements  is 


Baseline  Comparison  Performance  Shortfall  Worksheet  (PSH) 
W.Ksheet  #  _ 


Replace 


The 


relatively  straightforward.  The  required  technology 
information  could  be  obtained  directly  from  the  New  system 
equipment  description.  If  a  New  system  equipment  descrip¬ 
tion  is  not  available  from  the  contractor,  other  sources 
would  be  used  to  identify  the  new  technologies.  There  are 
two  major  sources  of  such  information:  (1)  equipment  related 
literature  and  documents,  and  (2)  interviews  with  subject 
matter  experts.  The  primary  source  for  the  identification 


of  new  technologies,  and  the  source  which  will  probably 
prove  to  be  the  easiest  to  access,  are  interviews  with 
subject  matter  experts  who  have  expertise  in  each  of  the  new 
technology  areas  associated  with  the  BCS  equipments. 

More  specifically,  the  Baseline  Comparison  Performance 
Shortfall  Worksheet  with  columns  (a)  thru  (e)  filled  in 
should  be  presented  to  hardware  subject  matter  experts  from 
the  functional  areas  associated  with  each  major  BCS 
subsystem.  These  analysts  should  then  be  asked  to  indicate 
what  new  technologies  can  be  added  to  the  BCS  equipment  to 
eliminate  the  performance  gaps.  Once  identified,  the  new 
technologies  should  be  entered  into  column  (f). 

1.5.3  Document  Data 

Information  on  new  technologies  should  be  documented  in  the 
worksheet  listed  in  Figure  1-19. 

1.6  FORMAT  NEW  SYSTEM 

OVERVIEW 


During  this  step,  the  new  technologies  to  be  incorporated  in 
the  New  system  design! s)  must  be  examined  and  differences 


between  the  BCS  and  the  New  system  which  impact  human  task 
performance  must  be  identified.  These  design  differences 
will  be  analyzed  in  detail  in  subsequent  procedures  and  used 
to  modify  BCS  data  to  reflect  the  infusion  of  new  tech¬ 
nologies  identified  in  Procedure  1.5. 

PROCEDURE 

An  overview  of  the  steps  comprising  this  procedure  is 
presented  in  Figure  1-20. 

1.6.1  Describe  Design  Differences 

A  design  difference  (DD)  worksheet  has  been  developed  to 
assist  the  analyst  in  describing  the  equipment  associated 
with  each  design  difference  (see  Figure  1-21). 

The  information  in  columns  (a)  and  (b),  describing  the  BCS 
equipments,  can  be  taken  directly  from  columns  (a)  and  (b) 
of  the  BCS  EQP  worksheet.  After  these  two  columns  are 
filled  in,  column  and  listing  the  New  system  equipment 
associated  with  each  BCS  equipment  should  be  filled  in. 
These  New  system  equipments  are  developed  by  combining  the 
new  technologies  listed  in  column  (f)  of  the  PSH  worksheet 
with  the  BCS  equipments.  Once  the  New  system  equipments 
have  been  identified,  subject  matter  experts  who  are 
familiar  with  the  BCS  equipments  should  be  presented  with 
the  partially  completed  DD  worksheet  and  the  completed  PSH 
worksheet  and  asked  to  identify  and  describe  any  potential 
hardwa re/so f tware  differences  between  the  BCS  and  New  system 
equipments  which  might  impact  the  task  performance  of 
operators,  maintainers,  or  support  personnel.  If  possible, 
they  should  be  asked  to  assign  a  quantitative  value  to  these 
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design  differences.  The  subject  matter  experts  should  b 
asked  to  focus  on  hardware/software  featui  :-s.  (for  exampl. 
reliability,  maintainability). 

1.6.2  Describe  New  System  Hardware/Software 

During  this  procedure,  a  hierarchical  equipment  structur 
for  the  New  system  is  constructed.  The  Design  Different 
worksheet  described  in  Figure  1-21  will  contain  a  listing  o 
New  system  equipments.  These  equipments  must  be  examine 
and  placed  in  a  hierarchical  equipment  structure.  If  a  Ne’ 
equipment  design  is  available  from  the  contractor  thi 
equipment  structure  can  be  taken  directly  from  contracto 
supplied  data. 

After  the  New  system  equipment  structure  is  identified,  th< 
New  system  equipments  may  be  described  in  more  detail  'sei 
Figure  1-22)  in  a  New  system  Equipment  Description  Work 
sheet.  Many  of  the  data  elements  on  the  worksheet  may  no 
be  available  until  later  in  the  acquisition  process  (see  tlm 
data  source  index  in  Appendix  E  for  a  listing  of  potentia 
data  sources )  . 

1.6.3  Document  Data 

Relevant  data  collected  in  this  procedure  must  be  enterei 
into  the  SDT.  This  data  shall  be  entered  as  the  workshee 
contained  in  Figure  1-22  is  completed.  Table  1-13  describe 
the  sequence  with  which  these  data  should  be  entered  inti 
the  SDT.  Additional  guidelines  for  entering  data  into  th 
SDT  are  contained  in  the  SDT  User's  Guide. 
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Figure  1-21  Design  Difference  Worksheet  (DD) 
Worksheet  #  - 


Procedure  1.4 


Note:  Items  (e)  through  (i)  are  optional 


Comparable  Equipment  Equipment  ±  (opt. 


SECTION  2.0 


GENERATE  TASKS 


OVERVIEW 

Dur  .ng  this  procedure,  task  lists  are  generated  for  the 
Baseline  Comparison  System  (BCS)  and  the  New  System.  In 
addition,  the  tasks  in  each  of  these  systems  are  assigned  to 
duty  position,  skill  level,  and  MOS. 

PROCEDURE 

This  procedure  consists  of  five  lower  level  procedures  (see 
Figure  2-1).  During  the  first  procedure,  relevant  task  data 
from  the  Predecessor  system  and  other  comparable  existing 
systems  is  collected.  In  the  second  procedure,  the  BCS 
tasks  are  generated.  In  the  third  procedure,  the  BCS  tasks 
are  assigned  to  duty  position,  skill  level,  and  MOS.  In  the 
fourth  procedure,  a  New  system  task  list  is  generated. 
Finally,  in  the  fifth  procedure,  the  New  system  tasks  are 
assigned  to  duty  position,  skill  level,  and  MOS. 

2.1  IDENTIFY,  COLLECT  AND  FORMAT  TASK  DATA 

OVERVIEW 


During  this  procedure,  task  data  are  collected  for  all 
Military  Occupational  Specialties  (MOSs)  which  may  be 
relevant  to  the  BCS  or  New  system. 
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PROCEDURE 


An  overview  of  the  steps  in  this  procedure  is  presented  in 
the  flowchart  in  Figure  2-2.  A  worksheet  which  can  be  used 
to  support  this  procedure  is  provided  in  Figure  2-3  (CND). 
There  are  two  steps  in  the  procedure.  In  the  first  step, 
candidate  occupational  specialities  are  identified.  In  the 
second  step,  task  data  on  these  occupational  specialties  are 
collected  and  formatted. 

2.1.1  Identify  MOSs  Currently  Performing  Similar  Functions 

The  purpose  of  this  step  is  to  identify  all  existing 

occupational  specialties  which  may  be  relevant  to  the  New 
system. 

To  begin  the  step,  information  on  the  new  equipment  number 
and  name  should  be  entered  into  columns  (a)  and  (b)  of  the 
CND  worxsheet.  This  information  can  be  obtained  from 
columns  (e)  and  (f)  of  the  BCS  worksheet  (see  Section 

1.4.1)  . 

Second,  if  the  BCS  equipment  is  in  the  current  Army 

inventory,  this  information  should  be  noted  in  column  (c). 

Third,  the  occupational  specialties  which  currently  operate 
and  maintain  the  BCS  equipment  should  be  entered  into  column 
(d).  Information  on  operator  MOSs  should  be  readily 
available  from  the  schools  associated  with  BCS  equipment. 
Information  on  maintenance  MOSs  can  be  obtained  from  the 
maintenance  manuals  associated  with  the  BCS  equipment. 
Information  on  Navy  and  Air  Force  occupational  specialties 
can  be  obtained  from  the  Navy  Enlisted  Manpower  and 
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Figure  2-3.  Candidate  MOS  Worksheet  (CND) 


Personnel  Classifications  and  Occupational  Standards  and  Air 
Force  Regulation  39-1  (see  Appendix  E  for  a  more  complete 
listing  of  data  sources). 

2.1.2  Collect  and  Catalog  Candidate  MOS  Task  Data 

During  this  step,  all  available  task  data  on  the  candidate 
MOSs  identified  in  step  2.1.1  must  be  collected.  A  listing 
of  potential  sources  of  task  data  is  provided  in  Table  2-1 
(a  more  complete  list  is  provided  in  Appendix  E).  The  best 
sources  of  task  data  for  existing  Army  systems  are  Soldier's 
Manuals  and  the  Trainer's  Guides. 

As  task  data  is  collected,  it  should  be  documented  in  column 
(e)  of  the  CND  worksheet  listed  in  Figure  2-3  and  in  the 
library  index. 

2.2  DEVELOP  BASELINE  COMPARISON  SYSTEM  ( BCS )  TASK  LIST 

OVERVIEW 

During  this  procedure,  a  task  list  is  generated  for  the 
Baseline  Comparison  System.  The  BCS  is  composed  of  the 
existing  subsystems  (including  the  predecessor  subsystems) 
which  come  closest  to  meeting  the  New  system  functional 
requirements  (see  Section  1.4  for  a  more  detailed 
description ) . 

PROCEDURE 

An  overview  of  the  steps  in  this  procedure  is  presented  in 
Figure  2-4.  A  worksheet  which  can  be  used  to  support  this 
procedure  is  provided  in  Figure  2-5  (BCST). 
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Table  2-1.  Task  Data  So  roes. 


•  Soldier  Manuals** 

•  Trainer's  Guide** 

•  Job  Book** 

•  CODAP** 

•  LSAR* 

•  ARTEPs** 

•  Field  Manuals** 

•  Maintenance  Manuals** 

•  Skill  Performance  Aids** 

•  Maintenance  Allocation  Charts* 

•  Training  Development  Information  System  (TDJS)** 

•  Contractor  Supplied  Training  Programs* 

•  Program  of  Instruction  (POI) 

*Data  source  for  developing  system 

**Data  source  for  existing  system 


BCS 

HARDWARE/ 

SOFTWARE 

UST 


SELECT  L 
SUBSYSTEM 
FROM  BCS 
HARDWARE/ 
SOFTWARE  UST 


BCS 

HARDWARE/ 

SOFTWARE 

SUBSYSTEM 


IDFNTIFY  Li! 
NON-ARMY 
OCCUPATIONAL 
SPECIALTY 
ASSOCIATED 
WITH  SUBSYSTEM 


|  2.2.3 

IDENTIFY 

MOS(iJ 

ASSOCIATED  WITH 
EQUIPMENT 


OCCUPATIONAL 

SPECIALTY 


RELEVANT 

MOS(i) 


Figure  2-4.  Develop  BCS  Task  List  (2.2). 
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Figure  2-5.  BCS  Task  Association  Worksheet  (BCST). 


2.2.1  Select  Subsystem  From  BCS  Hardware/Software  List 

To  begin  the  procedure,  the  BCS  equipment  number  and  name 
should  be  entered  into  columns  (a)  and  (b)  of  the  BCST 
worksheet  (This  information  can  be  obtained  from  columns  (e) 
and  (f)  of  the  BCS  worksheet  in  Section  1.4.1). 

Once  this  information  is  entered,  you  must  determine  if  the 
BCS  equipment  is  currently  in  the  Army  inventory.  (This 
information  is  available  in  column  (g)  of  the  CND  worksheet 
used  in  Procedure  2.1).  If  the  BCS  equipment  is  in  the  Army 
inventory,  you  should  proceed  to  step  2.2.3.  If  the  BCS 
equipment  is  not  in  the  Army  inventory,  you  should  proceed 
to  step  2.2.2. 

2.2.2  Identify  Non-Army  Occupational  Specialty 

Information  on  non-Army  occupational  specialties  can  be 
obtained  from  the  Navy  Enlisted  Manpower  and  Personnel 
Classifications  and  Occupational  Standards  and  from  Air 
Force  Regulation  39-1.  A  more  complete  list  of  data 
sources  is  provided  in  Appendix  E.  Once  the  occupational 
specialty  is  identified,  it  should  be  listed  in  column  (c) 
of  the  BCST  worksheet. 

2.2.3  Identify  MOSs  Associated  With  Equipment 

The  best  data  source  for  linking  MOS  to  equipment  is  the 
Table  of  Equipments  Manpower  Authorization  Criteria  (MACRIT) 
found  in  army  Regulation  AR  570-2.  However,  new  equipments 
are  not  likely  to  be  listed  in  MACRIT.  For  new  systems,  you 
must  identify  the  proponent  for  the  equipment/subsystem  or 
the  system  to  which  the  equipment/subsystem  belongs.  Once 


the  system  proponent  has  been  identified,  you  must  call  the 
proponent  school  (see  Table  1-3  in  Section  1.0)  to  obtain 
MOS  information.  The  most  definitive  data  source  for  ident¬ 
ifying  MOSs  for  Army  equipments  is  Army  Regulation  611-201, 
Enlisted ^Career  Management  Field  and  Military  Occupational 
Specialties.  This  document  describes  the  general  types  of 
activities  performed  by  each  Army  MOS. 

2.2.4  Identify  Comparable  Army  Equipment 

Comparable  Army  equipment  must  be  identified  for  each  BCS 
equipment  that  is  not  in  the  Army  inventory.  Procedures  for 
identifying  comparable  equipment  are  listed  in  Section 
1.4..  Once  identified,  the  comparable  equipment  should  be 
listed  in  column  (cl)  of  the  BCST  worksheet  and  occupational 
specialties  for  these  equipments  should  be  identified  using 
information  listed  in  Step  2.2.3. 

2.2.5  Identify  Tasks  Associated  With  Equipment 

The  tasks  associated  with  each  BCS  equipment,  which  is  in 

the  Army  inventory,  should  be  obtained  from  the  Army  task 
data  sources  listed  in  Appendix  E  (This  task  data  should 

have  been  collected  during  procedure  2.1).  Most  of  the 

time,  these  tasks  should  be  used  in  the  BCS  task  list 
without  modification.  However,  before  including  these  tasks 
in  the  BCS,  they  should  be  exafmined,  and  when  necessary, 
updated  to  reflect  any  equipment  or  doctrinal  changes  that 
were  made  since  the  task  data  was  published.  A  list  of 

sources  for  monitoring  these  changes  is  presented  in  Table 


Table  2-2.  List  of  Data  Sources  for  Monitoring 
Equipment/Doctrinal  Changes  for 
Existing  Systems* 


-  Tables  of  Organization  and  Equipment  (TOE) 

-  Tables  of  Distribution  and  Allowances  (TDA) 

-  Job  Supervisor  Interview 

-  Subject  Matter  Experts  (SME) 

-  Equipment  Modification  Work  Orders 

-  Directorate  of  Combat  Development 

(Organization  and  Operations  Concept) 

-  DA/TRADOC  Circulars  and  Pamphlets 

-  Army  Regulations 

-  Technical  Manuals 

-  Soldier's  Manuals 


♦Derived  from  Schulz  and  Farrel  (1980) 


Similarly,  if  the  BCS  equipment  is  not  in  the  Army 
inventory,  task  information  should  be  obtained  from  the 
appropriate  non-Army  data  source  (see  Appendix  E  for  a 
listing  of  these  data  sources).  Input  for  this  process  is 
provided  by  non-Army  occupational  specialties  listed  in 
column  (c)  of  the  BCST  worksheet.  As  with  the  Army  tasks, 
the  non-Army  tasks  should  be  updated  to  reflect  any 
equipments  or  doctrinal  changes  that  were  made  since  the 
task  data  was  published. 

2.2.6  Assess  Adequacy  of  Task  Descriptions 

Before  entering  the  task  statements  obtained  from  existing 
data  sources  into  the  BCST  worksheet  and  the  SDT,  the  user 
must  assess  the  instructional  adequacy  of  these  task 
statements.  Instructional  adequacy  refers  to  the  extent  to 
which  the  task  data  has  been  developed  in  accordance  with 
existing  Instructional  System  Development  (ISD)  processes. 
Table  2-3  displays  criteria  for  assessing  instructional 
adequacy  as  well  as  some  data  sources  for  obtaining  more 
detailed  guidance  on  assessing  the  adequacy  of  task 
descriptions.  As  indicated  in  the  introduction  to  this 
guide,  it  is  expected  that  ETES  users  are  thoroughly 
familiar  with  ISD  and  therefore  know  what  constitutes  a 
"good"  task  statement.  Hence,  detailed  instructions  on 
constructing  task  statements  are  not  provided  in  this  guide. 

If  the  instructional  adequacy  of  a  task  statement  is  good, 
you  can  enter  the  task  statement  into  column  (k)  of  the  BCST 
worksheet. 

If  a  task  statement  is  not  adequate,  you  must  generate  a  new 
statement  for  that  task  (See  Step  2.2.7  below). 


TABLE  2-3  CRITERIA  AND  SOURCES  FOR  ASSESSING 
INSTRUCTIONAL  ADEQUACY  OF  TASK  DATA 


ive  Authoring  flowcharts  for  Phase 
of  ISD  (Schulz  and  Farrell,  1980). 


2.2.7  Revise  Task  Descriptions 

Inadequate  task  statements  must  be  revised  in  accordance 
with  existing  ISD  procedures.  Guidelines  for  the  develop¬ 
ment  of  task  statements  can  be  found  in  TRADOC  PAM  350-30, 
Phase  I,  Section  1.2.3.  Additional  guidelines  are  provided 
in  the  data  sources  listed  in  Table  2-2.  Once  the  task 
statements  have  been  revised  they  should  be  entered  into 
column  (k)  of  the  BCST  worksheet. 

After  the  task  statements  have  been  entered,  task  numbers 
should  be  assigned  and  entered  into  column  (j)  of  the  BCST 
worksheet.  Table  2-4  provides  some  general  guidelines  for 
constructing  task  numbers.  More  detailed  guidance  is 
provided  in  TRADOC  Cir  351-28  and  AR  351-183. 

2.2.8  Enter  Data  Into  Data  Base 

Relevant  data  collected  in  this  procedure  must  be  entered 
into  the  SDT .  Table  2-5  describes  the  sequence  with  which 
these  data  should  be  entered  into  the  SDT.  Additional 
guidelines  for  entering  data  into  the  SDT  are  contained  in 
the  SDT  User's  Guide.  Remember  that  separate  SDT  data  bases 
should  be  maintained  for  the  BCS  and  the  New  System. 

2.3  ASSIGN  BASELINE  COMPARISON  SYSTEM  (BCS)  TASKS  TO  MOS, 
DUTY  POSITION,  AND  SKILL  LEVEL 

OVERVIEW 

During  this  procedure,  the  BCS  tasks  are  assigned  to  MOS, 
duty  position,  and  skill  level.  Comparability  analysis  is 
used  as  the  primary  mechanism  for  making  these  assignments. 


Table  2-4.  Guidelines  for  Assigning  Task  Numbers 


Enlisted  Task  Numbers  {10  digits)  -  XXX-YYY-ZZZZ 


XXX  -  Responsible  Agency  Code 

YYY  -  System  Subject  Code 

ZZZZ  -  Unique  Sequence  No.  used  by  responsible 
agency  to  identify  each  task 


Officer  Task  Numbers  (12  digits)  -  V-W-XXXX-YY-ZZZZ 

V  -  Current  Disposition  Indicator 
W  -  Prefix  (1-Commissioned,  2-Warrant,  3-Both) 
XXXX  -  Basic  Subject  Area 

YY  -  Additional  Subject  Area 


ZZZZ  -  Unique  Sequence  No. 


Comparability  analysis  provides  rough-cut  assignments  that 
are  only  appropriate  for  the  earliest  phases  of  the 
acquisition  process. 1 

PROCEDURE 

An  overview  of  the  steps  in  this  procedure  is  provided  in 
Figure  2-6.  During  the  first  step,  a  duty  position 
structure  is  identified  for  the  BCS.  In  the  second  step, 
the  BCS  tasks  are  assigned  to  duty  position  and  skill 
level.  In  the  third  step,  duty  positions  are  assigned  to 
Army  occupational  specialties.  In  the  fourth  step,  the  MOS, 
duty  position,  and  skill  level  assignments  are  reviewed  to 
take  into  account  any  manpower  or  human  factors  analyses 
which  may  have  been  conducted  during  the  early  phases  of 
the  acquisition  program.  The  BCST  worksheet  used  in 

Procedure  2.2  to  describe  BCS  tasks  is  also  used  in  this 
procedure  to  describe  MOS,  duty  position,  and  skill  level 
assignments. 

2.3.1  Identify  Duty  Position  Structure 

A  duty  position  structure  may  be  defined  as  a  listing  of  the 
number  and  type  of  each  duty  position  associated  with  a 
system.  Construction  of  the  BCS  duty  position  structure 
occurs  in  two  steps.  First,  the  duty  positions  structure 
for  the  predecessor  system  are  identified.  Second,  duty 
positions  are  added  or  deleted  from  the  predecessor  list  to 
reflect  the  equipments  that  were  added/deleted  to  the 
predecessor  system  to  form  the  BCS  system. 


1  A  description  of  comparability  analysis  procedures  is 
provided  in  O'Brien  (1983). 
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Figure  2-6.  Overview  Diagram:  Assign  BCS  Tasks  to  MOS, 
Duty  Position,  and  Skill  Level  (2.3). 


2.3.2  Assign  Tasks  To  Duty  Position  and  Skill  Level 

Assignment  of  tasks  to  duty  position  and  skill  level  is  a 
three-step  process.  First,  the  BCS  task  list  must  be 
examined  to  identify  any  tasks  which  are  associated  with 
duty  positions  (or  billets  if  a  Navy  system  is  involved) 
which  are  not  in  the  BCS  duty  position  structure  identified 
in  2.3.1.  The  user  must  then  identify  what  BCS  duty 
positions  would  be  most  likely  to  perform  the  tasks 
associated  with  each  of  these  other  duty  positio-s.  This  is 
accomplished  by  comparing  the  tasks  already  a.‘  ;Cxated  with 
the  BCS  duty  positions  and  the  tasks  associ  _J  with  the 
other  duty  positions. 

Second,  each  task  should  be  assigned  to  a  BCS  duty  position. 
If  a  task  is  already  associated  with  a  BCS  duty  position, 
then  that  duty  position  should  be  assigned  to  the  task.  If  a 
task  is  not  currently  assigned  to  a  BCS  duty  position,  it 
should  be  assigned  to  the  BCS  duty  position  which  is  most 
similar  to  the  duty  position  from  which  it  was  derived. 

Third,  each  BCS  duty  position  should  be  assigned  a  skill 
level.  This  is  accomplished  by  identifying  similar  duty 
positions  in  the  predecessor  or  other  similar  systems  and 
assigning  the  BCS  duty  position  the  skill  level  associated 
with  that  duty  position. 

2.3.3  Assign  Tasks  to  MOS 


Tasks  are  assigned  to  an  MOS  by  comparing  the  tasks 
associated  with  the  duty  position  to  the  task  requirements 
of  existing  MOS  that  are  relevant  to  the  new  system.  Task 
requirements  for  MOSs  can  be  identified  by  examining  (a)  AR 


611-201  which  describes  the  occupational  specialties  and  (b) 
the  Soldier's  Manual  for  each  specialty  which  provides 
detailed  data  on  MOS  tasks. 

If  none  of  the  MOSs  listed  in  AR  611-201  match  the  task 
requirements  of  the  BCS,  it  may  be  necessary  to  establish 
requirements  for  a  new  MOS.  Whenever  possible,  an  existing 
MOS  should  be  used  since  the  cost  of  establishing  a  new  MOS 
is  significant.  New  MOSs  should  be  designated  by  the  terms 
XI,  X2,  X3,  etc.  in  place  of  the  two  digits  normally  used  to 
designate  an  MOS.  More  details  on  the  requirements  for 
establishing  a  new  MOS  are  found  in  AR  611-1. 

2.3.4  Assess  Integrated  Impacts  of  Assignments 

Concurrent  with  an  early  training  estimation  study,  there 
may  be  additional  studies  assessing  the  system  manpower  or 
human  factors  requirements . 2  If  there  are  such  studies,  the 
MOS,  duty  position,  and  skill  level  assignments  developed  in 
the  previous  step  should  be  updated  to  reflect  the  more 
detailed  input  that  these  manpower  and  human  factors  studies 
typically  provide.  (Such  studies  typically  assess  the 
workload  of  operators  and  maintainers  and  this  workload  data 
provides  more  detailed  information  on  what  tasks  can  be 
performed  by  each  duty  position.) 

After  the  BCS  skill  level,  duty  position,  and  MOS 
assignments  have  been  finalized,  they  should  be  entered  into 
columns  (e),  (m),  and  (n)  of  the  BCST  worksheet, 
respectively. 


2  At  the  present  time,  such  studies  are  seldom,  if  ever, 
conducted  during  the  early  phases  of  the  acquisition 
process . 
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2.3.5  Document  Data 

Relevant  data  collected  or  developed  in  this  procedure  must 
be  entered  into  the  SDT.  Table  2-6  describes  the  sequence 
with  which  these  data  should  be  entered  into  the  SDT. 
Additional  guidelines  for  entering  data  into  the  SDT  are 
contained  in  the  SDT  User's  Guide.  Remember  that  separate 
SDT  data  bases  should  be  maintained  for  the  BCS  and  New 
system. 

2.4  DEVELOP  NEW  SYSTEM  TASK  LIST 

OVERVIEW 

During  this  procedure,  a  task  list  is  generated  for  the  New 
system.  This  is  accomplished  by  adding,  deleting,  or 
modifying  BCS  tasks. 

PROCEDURE 

An  overview  of  the  steps  in  this  procedure  is  presented  in 
the  flowchart  in  Figure  2-7.  A  worksheet  (NTA)  for 
documenting  the  procedure  is  provided  in  Figure  2-8. 

To  begin  the  procedure,  the  new  technologies  identified  in 
column  (f)  of  worksheet  PSH  as  part  of  Procedure  1.5  should 
be  entered  in  column  (a)  of  the  New  System  Task  Generation 
Worksheet  (NTA).  The  New  system  equipment  list  should  then 
be  entered  into  Columns  (b)  and  (c)  of  the  NTA  worksheet. 
Information  on  New  equipment  names  and  numbers  can  be 
obtained  from  columns  (a)  and  (b)  of  the  Baseline  Comparison 
System  Performance  Shortfall  Worksheet  (PSH). 
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It  is  only  necessary  to  analyze  tasks  for  those  subsystems 
which  are  associated  with  design  impacts.  For  all  other 
subsystems,  the  baseline  tasks  can  be  used  without 
modification. 

A  task  list  can  be  generated  for  the  subsystems  related  to 
design  impacts  by  adding  or  deleting  tasks  from  the 
corresponding  BCS  subsystem.  These  additions  or  deletions 
should  be  documented  using  the  codes  listed  in  Table  2-7, 
More  specifically,  the  BCS  task  data  should  be  examined  and 
compared  to  the  New  system  hardware/software  design  and 
functional  requirements.  BCS  tasks  associated  with 
equipments  or  functions  which  are  no  longer  needed  should  be 
eliminated.  New  tasks  should  be  added  to  reflect  new 
equipment  or  functions.  New  functions  can  be  identified  by 
examining  the  functional  requirements  generated  during 
Procedure  1.3.  An  overview  of  the  relationship  between 
subsystem  functions  and  individual  tasks  is  presented  in 
Figure  2-9.  As  Figure  2-9  indicates,  individual  tasks  can 
be  viewed  as  lower  level  functions  and  these  functions  can 
provide  a  top-down  perspective  on  task  identification. 

Identification  of  New  system  maintenance  tasks  can  be 
facilitated  by  examining  the  standard  list  of  action  verbs 
that  are  used  to  describe  maintenance  tasks.  A  listing  of 
these  maintenance  action  verbs  is  presented  in  Table  2-8. 
As  part  of  the  system  development,  a  Maintenance  Allocation 
Chart  (MAC)  may  be  constructed  for  the  New  system.  If  a  MAC 
is  available,  it  can  probably  be  obtained  from  the  New 
system  program  office.  The  MAC  is  an  excellent  source  of 
task  data  since  it  describes  which  of  the  standardized  list 
of  maintenance  actions  will  be  performed  on  the  system  and 
at  what  levels  of  maintenance  these  actions  will  be 
performed.  Information  on  the  MOSs  performing  each  action  is 
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TABLE  2-7  TASK  DELETION/MODIFICATION/ADDITION  CODES 


TASK  DELETION 
Code 


ELI 

AUT 


RTF 

MC 

OC 


TASK  MODIFICATION 
Code 


NC 

MIN 


SKI 


FRE 


MAJ 


TASK  ADDITION 
Code 


* 


Reason  for  Deletion 


Elimination  of  subsytem 

Task  Automation  -  task  now  performed 
by  equipment 

Reduced  task  frequency 

Change  in  maintenance  concept/doctrine 

Change  in  operational  concept/doctrine 


Type  of  Modification 


No  change  in  task 

Minor  task  modification  -  task 
essentially  the  same.  Only  minor 
change  in  equipment/procedure  required. 


Skill  level  change  -  task  essentially 
the  same  but  assigned  to  different 
skill  level. 


Frequency  change  -  Same  task  but  task 
is  performed  more  (less)  frequently 
due  to  change  in  reliability,  etc. 


Major  task  modification  -  Significant 
change  in  skills  and  knowledges  and/or 
other  task  characteristics. 


Reason  for  Addition 


[V. 

h> 

ADD 

Addition 

of  subsystem 

MAN 

Automated 

task  converted  to  manual 

tm 

ITF 

Increased 

task  frequency 

K-. 

AMC 

Change  in 

maintenance  concept/doctrine 

> 

AOC 

Change  in 

operational  concept/doctrine 

is 

IT 

Increase 

in  technology 

r. 
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TASK  ELEMENTS 


TABLE  2-8  MAINTENANCE  ACTIONS 


f  • 


4 


,v 


a.  INSPECT.  To  determine  the  serviceability  of  an  item 

by  comparing  its  physical,  mechanical,  and/or  electrical 
characteristics  with  established  standards,  through 
examination. 

b.  SERVICE.  Operations  required  periodically  to  keep  an 
item  in  proper  operating  condition;  i.e.,  to  clean 
(decontaminate),  to  preserve,  to  drain,  to  paint,  or  to 
replenish  fuel,  lubricants,  hydraulic  fluid  ,  or  compressed 
air  supplies. 


c.  TEST.  To  verify  serviceability  and  detect  incipient  failure 
by  measuring  the  mechanical  or  electrical  characteristics 
of  an  item  and  comparing  those  characteristics  with  pre¬ 
scribed  standards. 


[v 


d.  ADJUST.  To  maintain  within  prescribed  limits,  by  bringing 
into  proper  or  exact  position,  or  by  setting  the  operating 
characteristics  to  specified  parameters. 

e.  ALIGN.  To  adjust  specified  variable  elements  of  an  item 
to  bring  about  optimum  or  desired  performance. 

f.  CALIBRATE.  To  determine  and  cause  corrections  to  be  made 
or  to  be  adjusted  on  instruments  or  test  measuring  and 
diagnostic  equipment  used  in  precision  measurement.  Con¬ 
sists  of  comparisons  of  two  instruments,  one  of  which  is 
a  certified  standard  of  known  accuracy,  to  detect  and 
adjust  any  discrepancy  in  the  accuracy  of  the  instrument 
being  compared. 


g.  INSTALL.  The  act  of  emplacing,  seating,  or  fixing  into 
position  an  item,  part,  or  module  (component  or  assembly) 
in  a  manner  to  allow  the  proper  functioning  of  an  equipment 
or  system. 


N 

r. 


► 


V 
*  1 


h.  REPLACE.  The  act  of  substituting  a  serviceable  like  type 
part,  subassembly,  or  module  (component  or  assembly)  for 
an  unserviceable  counterpart. 

i.  REPAIR.  The  application  of  maintenance  services  or  other 
maintenance  actions  to  restore  serviceability  to  an  item 
by  correcting  specific  damage,  fault,  malfunction,  or 
failure  in  a  part,  subassembly,  module  (component  or 
assembly),  end  item  or  system. 


Lv.v.v 


aV. 
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TABLE  2-8  (continued) 


OVERHAUL.  That  maintenance  effort  (service/action) 
necessary  to  restore  an  item  to  a  completely  serviceable/ 
operational  condition  as  prescribed  by  maintenance 
standards  (i.e.,  DMWR)  in  appropriate  technical  publica¬ 
tions.  Overhaul  is  normally  the  highest  degree  of  main¬ 
tenance  performed  by  the  Army.  Overhaul  does  not  normally 
return  an  item  to  like  new  condition. 


REBUILD.  Consists  of  those  services/actions  necessary 
for  the  restoration  of  unserviceable  equipment  to  a  like 
new  condition  in  accordance  with  original  manufacturing 
standards.  Rebuild  is  the  highest  degree  of  materiel 
maintenance  applied  to  the  Army  equipment.  The  rebuild 
operation  includes  the  act  of  returning  to  zero  those 
age  measurements  (hours/miles,  etc.)  considered  in 
classifying  Army  equipmenty components . 
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often  included  in  the  MAC.  A  sample  MAC  is  provided  in 

Table  2-9.  If  a  MAC  is  not  available,  the  user  should 
consider  developing  a  preliminary  MAC  as  an  aid  to 
maintenance  task  identification.  MACs  from  comparable 
existing  systems  provide  a  starting  point  for  development  of 
a  preliminary  MAC. 

It  is  possible  that  a  New  subsystem  may  require  the  same 

task  as  a  BCS  subsystem.  The  essential  characteristics  of 
this  task,  however,  must  frequently  be  changed  to  reflect 
the  New  system  requirements.  For  example,  the  same  task  may 
be  required  for  both  the  BCS  and  New  system  but  the 

frequency  with  which  the  task  is  performed  may  vary 

significantly  between  the  BCS  and  New  systems.  The  task  may 
have  to  be  performed  with  greater  frequency  in  the  BCS  than 
in  the  New  system  due  to  use  of  superior  materials  and 
lubricants  in  the  New  system.  To  account  for  these  changes 
to  essential  task  characteristics,  all  tasks  with  major 
modifications  must  be  identified  and  the  reason  for  the  task 
modification  must  be  documented  using  the  code  listed  in 
Table  2-7.  Tasks  associated  with  major  changes  to  skills 
and  knowledges  (i.e.,  tasks)  in  the  MAJ  category  are 
analyzed  further  in  Procedure  3.2  to  identify  the  specific 
skill  and  knowledge  differences  between  the  BCS  and  New 
system  tasks.  Tasks  with  minor  changes  in  task  elements  are 
not  examined  further  (it  is  assumed  that  their  skills  and 
knowledges  have  not  changed).  The  task  addition,  deletion, 
and  modification  codes  should  be  entered  in  columns  (q)  , 
(r),  and  (s)  of  the  NTA  worksheet. 


Once  tasks  have  been  identified,  their  conditions  and 
standards  should  be  developed  using  existing  instructional 
systems  development  procedures  from  the  source  documents 
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specified  in  Table  2-3.  These  same  procedures  can  be  used 
to  assess  the  adequacy  of  any  new  task  descriptions  which 
are  generated  during  comparability  analysis. 

The  completed  New  system  list  should  be  presented  to  the 
Program  Office  and  cognizant  school  for  review. 

To  complete  this  procedure,  relevant  task  data  should  be 
entered  into  the  SDT.  Table  2-10  describes  the  sequence 
with  which  these  data  should  be  entered  into  the  SDT. 
Additional  guidelines  for  entering  data  into  the  SDT  are 
contained  in  the  SDT  User's  Guide.  Remember  that  separate 
SDT  data  bases  should  be  maintained  for  the  BCS  and  New 
system. 

2.5  ASSIGN  NEW  SYSTEM  TASKS  TO  MOS ,  DUTY  POSITION  AND  SKILL 
LEVEL 

OVERVIEW 

During  this  procedure,  the  New  system  tasks  are  assigned  to 
MOS,  duty  position,  and  skill  level.  Comparability  analysis 
is  used  as  the  primary  mechanism  for  making  these  assign¬ 
ments.  Comparability  analysis  provides  rough-cut  assignments 
that  are  only  appropriate  for  the  earliest  phases  of  the 
acquisition  process. 

PROCEDURE 

An  overview  of  the  steps  in  this  procedure  is  provided  in 
Figure  2-10.  During  the  first  step,  a  duty  position 
structure  is  identified  for  the  New  system.  In  the  second 
step,  the  BCS  tasks  are  assigned  to  duty  position  and  skill 


RELATED  RELATED  WORKSHEET 

PROCEDURE  DATA  ELEMENT  SDT  ENTITY  WORKSHEET  COLUMN (S)  SEQ  II 


Figure  2-10.  Overview  Diagram:  Assign  New  System  Tasks  to  MOS, 
Duty  Position,  and  Skill  Level  (2.5). 
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level.  In  the  third  step,  duty  positions  are  assigned  to 
MOS.  In  the  fourth  step,  the  MOS,  duty  position,  and  skill 
level  assignments  are  reviewed  to  take  into  account  any 
manpower  or  human  factors  analyses  which  may  have  been 
conducted  during  the  early  phases  of  the  acquisition 
program.  The  NTA  worksheet  used  in  Procedure  2.4  to 

describe  New  system  tasks  is  also  used  in  this  procedure  to 
describe  MOS,  duty  position,  and  skill  level  assignments. 

2.5.1  Identify  Duty  Position  Structure 

A  duty  position  structure  may  be  defined  as  a  listing  of  the 
number  and  type  of  each  duty  position  associated  with  a 

system.  Construction  of  a  New  system  duty  position 

structure  occurs  in  two  steps.  First,  the  duty  position  for 
the  BCS  system  is  examined.  Second,  duty  positions  are 
added  or  deleted  from  the  BCS  list  to  reflect  the  equipments 
or  functions  that  were  added/deleted  to  the  BCS  system  to 
form  the  New  system. 

2.5.2  Assign  Tasks  to  Duty  Position  and  Skill  Level 

Assignment  of  tasks  to  duty  position  and  skill  level 

involves  a  three  step  process.  First,  the  New  system  task 

list  must  be  examined  to  identify  any  tasks  which  are 
associated  with  duty  positions  (or  billets  if  a  Navy  system 
is  involved)  which  are  not  in  the  New  system  duty  position 
structure  identified  in  2.5.1.  The  user  must  then  identify 
what  New  system  duty  positions  would  be  most  likely  to 
perform  the  tasks  associated  with  each  of  these  other  duty 
positions.  This  is  accomplished  by  comparing  the  tasks 
already  associated  with  the  New  system  duty  positions  and 
the  tasks  associated  with  the  other  duty  positions. 
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Second,  all  tasks  should  be  assigned  to  a  New  system  duty 
position.  I£  the  task  is  already  associated  with  a  New 
system  duty  position,  then  that  duty  position  should  be 
assigned  to  the  task.  If  it  is  not  currently  assigned  to  a 
New  system  duty  position,  it  should  be  assigned  to  the  New 
system  duty  position  which  is  most  similar  to  the  duty 
position  from  which  it  was  derived. 

Third,  each  New  system  duty  position  should  be  assigned  a 
skill  level.  This  is  accomplished  by  identifying  similar 
duty  positions  in  the  predecessor  of  other  similar  systems 
and  assigning  the  New  system  duty  position  the  skill  level 
associated  with  that  duty  position. 

2.5.3  Assign  Tasks  to  MOS 

Tasks  are  assigned  to  MOSs  by  comparing  the  tasks  associated 
with  the  duty  position  to  the  task  requirements  of  existing 
MOSs  which  are  relevant  to  the  New  system.  Task  requirements 
for  MOSs  can  be  identified  by  examining  (a)  AR  611-201  which 
describes  the  MOSs  and  (b)  the  Soldier's  Manual  for  each 
specialty  which  provides  detailed  data  on  MOS  tasks. 

If  none  of  the  MOSs  listed  in  AR  611-201  match  the  task 
requirements  of  the  New  system,  it  may  be  necessary  to 
establish  requirements  for  a  new  MOS.  Whenever  possible,  an 
existing  MOS  should  be  used  due  to  the  significant  cost  of 
establishing  a  new  MOS.  New  MOSs  should  be  designated  by 
the  terms  XI,  X2,  X3  etc.  in  place  of  the  two  digits  nor¬ 
mally  used  to  designate  an  MOS.  More  details  on  the  require¬ 
ments  for  establishing  a  new  MOS  are  found  in  AR  611-1. 


2.5.4  Assess  Integrated  Impacts  of  Assignments 

Concurrent  with  an  early  training  estimation  study,  there 
may  be  additional  studies  assessing  the  system  manpower  or 
human  factors  requirements.  If  there  are  such  studies,  the 
MOS,  duty  position,  and  skill  level  assignments  developed  in 
the  previous  step  should  be  updated  to  reflect  the  more 
detailed  input  that  these  manpower  and  human  factors  studies 
typically  provide.  (Such  studies  typically  assess  the 
workload  of  operators  and  maintainers  and  this  workload  data 
provides  more  definite  information  on  what  tasks  can  be 
performed  by  each  duty  position). 

After  the  New  system  skill  level,  duty  position,  and  MOS 
assignments  have  been  finalized,  they  should  be  entered  into 
columns  (e),  (m),  and  (n)  of  the  NTA  worksheet, 
respectively. 

2.5.5  Document  Data 

Relevant  data  collected  or  developed  in  this  procedure  must 
be  entered  into  the  SDT.  Table  2-11  describes  the  sequence 
with  which  these  data  should  be  entered  into  the  SDT. 
Additional  guidelines  for  entering  data  into  the  SDT  are 
contained  in  the  SDT  User's  Guide.  Remember  that  separate 
SDT  data  bases  should  be  maintained  for  the  BCS  and  New 
system. 
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SECTION  3 


ESTIMATE  TRAINING  PROGRAM 


OVERVIEW 

During  this  procedure,  an  estimate  of  the  institutional 
training  program  is  developed  for  both  the  Baceline 
Comparison  System  (BCS)  and  New  System.  The  major  products 
of  this  procedure  are  (1)  quasi-program  of  instructions 
which  outline  the  contents,  methods,  and  media  to  be 
included  in  each  institutional  training  course  and  (2) 
training  paths  describing  the  sequence  with  which  these 
courses  must  be  taken  in  an  MOS. 

PROCEDURE 

This  procedure  consists  of  eight  lower  level  procedures  (See 
Figure  3-1).  During  this  first  procedure,  tasks  requiring 
training  are  identified.  During  the  second  procedure  the 
tasks  are  assigned  to  training  settings.  In  the  third 
procedure,  the  general  skills  and  knowledges  associated  with 
the  tasks  to  be  trained  are  identified.  In  the  fourth 
procedure,  a  target  population  description  is  developed.  In 
the  fifth  procedure,  the  sequence  in  which  the  tasks  will  be 
trained  is  identified.  In  the  sixth  procedure,  quasi¬ 
program  of  instructions  are  constructed.  In  the  seventh 
procedure,  the  automated  Media  Selection  Program  is  used  to 
assign  tasks  to  media.  In  the  eighth  procedure,  training 
paths  are  identified.  It  should  be  noted  that  the  present 
version  of  ETES  only  estimates  training  programs  for 
individual  institutional  training  courses. 


Figure  3-1  Overview  Diagram:  Estimate  Training  Program 
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3.1  SELECT  TASKS  FOR  TRAINING 


Once  a  task  inventory  has  been  compiled,  the  tasks  in  this 
inventory  must  be  analyzed  to  determine  which  tasks  require 
training.  Since  the  task  inventory  for  the  typical  MOS  is 
composed  of  several  hundred  tasks,  it  is  inappropriate  to 
even  consider  training  all  of  these  tasks.  Thus,  the  Army 
cannot  formally  train  the  individual  soldier  on  all  of  the 
tasks  required  for  full  job  proficiency.  Even  if  the 
resources  were  available  such  a  strategy  would  not  be  cost 
efficient  considering  the  relatively  low  rate  of  soldier 
reenlistment  and  career  progression. 

This  procedure  describes  how  to  select  tasks  for  training 
(or  conversely,  to  identify  tasks  which  will  not  require 
training).  This  procedure  provides  several  alternative 
methods  for  making  the  train-no  train  decision.  These 
methods  differ  in  terms  of  (1)  the  criteria  used  to  select 
tasks  for  training  and  (2)  the  techniques  used  to  aggregate 
the  scores  on  the  criterion  variables.  An  overview  diagram 
depicting  the  steps  contained  in  this  procedure  is  presented 
in  Figure  3-2.  Procedure  3.2,  Assign  Tasks  to  Training 
Settings,  will  assign  the  tasks  to  be  trained  to  training 
settings.  The  major  steps  in  the  task  selection  process 
include  choosing  a  task  selection  and  aggregation  method, 
obtaining  task  factor  ratings/scores,  applying  criterion/ 
cutoff  scores  to  develop  a  tentative  task  list  for  training, 
reviewing  these  tasks  before  making  final  task  selections, 
and  documenting  the  data  associated  with  the  procedure. 
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3.1.1  Choose  Task  Selection  and  Aggregation  Methods 

OVERVIEW 

There  is  no  single  task  selection  method  which  can  meet  the 
needs  of  the  different  phases  of  the  acquisition  process  and 
be  effectively  applied  across  all  occupational  specialties. 
Selection  of  tasks  for  training  requires  a  management 
decision  based  on  the  best  data  available.  A  number  of 
alternative  methods  have  merit  and,  depending  on  the 
characteristics  of  the  specialty  analyzed  and  the  require¬ 
ments  of  the  analysis,  may  be  appropriate  for  task  selec¬ 
tion.  A  summary  of  these  methods  is  provided  in  Table  3-1. 
An  overview  of  procedure  3.1.1  is  provided  in  Figure  3-3. 

PROCEDURE 

3. 1.1.1  Identify  Task  Selection  Requirements 

In  order  to  choose  a  task  selection  methodology  for  a 
particular  application  the  user  must  determine  the  (1) 
accuracy  requirements,  (2)  the  data  availability,  and  (3) 
resources  available  to  conduct  the  analysis. 

During  the  earliest  phases  of  system  acquisition,  when 
several  alternative  system  concepts  are  still  being  con¬ 
sidered,  a  high  degree  of  accuracy  may  not  be  required.  The 
optimal  approach  may  be  one  with  moderate  accuracy  and  low 
data  requirements,  that  is,  a  "quick  and  dirty"  approach. 

During  the  later  phases  of  system  acquisition,  when  the 
accuracy  of  results  is  critical  for  training  resource 
requirements  analysis,  more  data  intensive  and  accurate 
methods  may  be  preferable. 
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Thus,  the  determination  of  the  data  and  accuracy 
requirements  of  a  study  must  precede  the  set-  -tion  of  an 
approach  to  task  selection. 

3.  1.1.2  Choose  Task  Selection  Method 

Four  task  selection  methods  are  presented  in  Table  3-1. 
These  methods  are  described  in  terms  of  (1)  their  data 
requirements  and  (2)  their  estimated  accuracy.  For  example, 
comparability  analysis  has  the  fewest  data  requirements  and 
can  therefore  be  applied  early  in  the  acquisition  process. 
However,  the  accuracy  of  the  results  does  not  compare 
favorably  with  the  alternative  methods.  Thus,  comparability 
analysis  should  be  used  only  when  a  quick  and  inexpensive 
analysis  is  required  in  the  very  earliest  period  of  the 
acquisition  process.  The  other  methods  listed  in  Table  3-1 
have  progressively  increasing  data  requirements  and 
corresponding  improvements  in  analysis  accuracy.  These 
methods  are  described  in  the  paragraphs  which  follow. 


o  Comparability  Analysis 

Comparability  analysis  may  be  used  to  provide  crude  task 
training  assignments  during  the  early  phases  of  the 
acquisition  process.  The  first  step  in  the  application  of 
comparability  analysis  is  to  identify  a  comparable  existing 
task  for  each  of  the  tasks  in  the  New  System.  A  comparable 
existing  task  may  be  defined  as  a  task  which  (1)  is 
performed  on  comparable  equipment  or  (2)  involves  the  same 
type  of  task  action  (that  is,  the  action  verb  for  the 
comparable  task  matches  or  is  highly  similar  to  the  action 
verb  for  the  New  system  task). 


The  second  step  in  comparability  analysis  is  to  assign  the 
New  system  task  to  the  same  training  categories  (training, 
no  training)  assigned  to  their  comparable  tasks. 

o  DIF  Model1 

Another  strategy  for  selecting  tasks  for  training  is  a 
scheme  called  the  DIF  (Difficulty,  Importance,  Freguency) 
analysis.  This  methodology  is  a  simple  approach  to  a 
complex  problem,  in  that  it  integrates  several  different 
criteria  during  the  task  selection  process  separately.  The 
supervisor  and  incumbent  are  asked  to  determine  the 
difficulty  of  the  task  in  terms  of  learning  and  performing, 
then  asked  to  comment  on  the  importance  of  the  task,  and 
finally  to  estimate  the  frequency  of  task  performance. 

In  determining  the  "difficulty"  of  the  task,  these  responses 
encompass  both  task  learning  difficulty  and  probability  of 
deficient  performance.  For  the  decision  on  "importance"  the 
respondents'  conscious  or  unconscious  responses  may  well 
include  factors  such  as  the  sequences  of  inadequate 
performance,  the  task  delay  tolerance,  percent  performing, 
and  time  spent  performing.  When  estimating  the  "frequency", 
both  frequency  data  and  the  time  between  job  entry  and  task 
performance  might  be  considered. 

This  approach  to  task  selection  implies  that  asking  the 
supervisor  and  the  job  incumbent  three  simple  questions  will 
facilitate  the  task  selection  process.  This  approach  has 


1  The  descriptions  of  the  DIF,  Four  Factor  Model  and 
Eight  Model  presented  in  this  section  were  derived  from 
TRADOC  PAM  351-4. 


merit  because  of  its  simplicity  yet  degree  of  precision  it 
may  achieve.  The  survey  would  be  conducted  by  the  service 
school  (field  surveys)  and  requires  a  relatively  small 
sample  (40  incumbents  and  40  supervisors).  A  graphic 
portrayal  of  this  sequence  is  described  in  Figure  3-4. 

After  going  through  this  branching  process  it  can  be  seen 

that  at  one  end  of  the  scale  are  different,  important,  and 
infrequently  performed  tasks.  Each  of  these  instances  has  a 
different  rationale  for  whether  to  train  the  tasks.  The 

sophistication  and  sensitivity  of  this  technique  can  be 
increased  by  incorporating  degrees  of  importance, 
difficulty,  and  frequency,  and  by  using  more  detailed  levels 
of  training  as  shown  in  Figure  3-5. 

o  4-Factor  Model 

The  4-Factor  Model  for  task  selection  consists  of  four 

criteria  for  task  selection  that  might  influence  the 

selection  of  tasks  for  training.  These  criteria  are: 

o  Percent  performing, 

o  Task  delay  tolerance, 

o  Task  learning  difficulty,  and 

o  Consequences  of  inadequate  performance. 

Information  on  these  four-factors  is  collected  from  job 
holders.  Information  can  be  collected  directly  by  the 
analyst  or  from  surveys  administered  by  the  Army 
Occupational  Survey  Program  (AOSP)  of  MILPERCEN.  Once 
information  has  been  collected  for  each  of  the  four  factors 
for  every  task  on  the  task  inventory,  there  will  be  a  large 
data  base  to  use  for  selecting  tasks  for  training.  These 
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Figure  3-4  DIF  Model 
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Figure  3-5  Detailed  DIF 


factors  are  summarized  in  Table  3-2  and  are  described  in 
more  detail  in  Procedure  3.1.2. 

o  8-Factor  Model 

The  8-Factor  Model  contains  eight  criteria  including  those 
comprising  the  4-Factor  Model.  The  additional  four  criteria 
are : 

o  Percent  of  time  spent  performing, 

o  Frequency  of  performance, 

o  Probability  of  deficient  performance,  and 

o  Immediacy  of  performance  or  time  between  job  entry 
and  task  performance. 

As  in  the  4-Factor  Model,  information  can  be  collected  from 
job  holders  directly  by  the  analyst  or  from  AOSP  surveys. 
The  8-Factor  Model  provides  a  more  comprehensive  basis  for 
selecting  tasks  for  training  than  does  the  4-Factor  Model, 
but,  as  stated  earlier,  requires  more  extensive  data 
collection.  The  factors  listed  above  are  described  in  more 
detail  in  Procedure  3.1.2. 

Selecting  tasks  for  training  is  probably  the  most  important 
management  decision  made  early  in  training  design.  The 
tasks  selected  for  training  (1)  provide  the  specialty  base, 
(2)  drive  the  production  of  training  materials.  Soldier's 
Manuals,  SQTs ,  and  (3)  are  the  basis  of  the  specialization 
within  each  specialty.  If  all  tasks  cannot  be  trained,  then 
the  tasks  that  are  trained  must  be  the  right  ones  in  terms 
of  job  proficiency,  transferability,  and  combat  criticality. 
Each  of  these  models  described  above  requires  a  different 
amount  of  analysis  effort  and  time.  Each  model  has  certain 


Table  3-2.  Data  Sources  for  4-Factor  Model 


Task  Selection  Factor 

-  Percent  Performing 

-  Task  Delay  Tolerance 

-  Task  Learning  Difficulty 

-  Consequences  of  Inadequate 
Performance 


Source 

Percent  of  soldiers  surveyed 
who  indicate  they  perform  the 
task 

Average  of  responses  from  l(low) 
-  7 (high)  indicated  by  soldiers 
surveyed  who  perform  the  task 


advantages  and  limitations  such  as  ease  of  use,  accept¬ 
ability  by  field,  comprehensiveness,  timeliness,  etc. 


Based  on  the  data  requirements  of  the  alternative  selection 
methods,  the  accuracy  requirements  of  the  task  selection 
process,  and  an  understanding  of  the  differences  among  the 
alternative  procedures,  the  analyst  must  choose  a  task 
selection  method. 

3. 1.1. 3  Select  Aggregation  Method 

In  each  alternative  task  selection  procedure,  some  method 
must  be  used  to  aggregate  data.  Thus,  in  the  eight  factor 
model  for  example,  ratings  on  eight  criteria  have  been 
obtained  for  each  task.  These  ratings  must  be  combined  to 
yield  a  single  measure  of  the  requirement  for  training. 

Because  of  the  comprehensiveness  and  complexity  of  the 
factors,  it  is  not  easy  to  simply  combine  all  factors  and 
pick  the  most  important  tasks  for  training.  One  aggregation 
strategy  requires  the  command  to  designate  the  most 
important  factor(s).  A  value  is  specified  as  a  cutoff  point 
for  that  factor.  If  a  task  is  above  that  cutoff  point  on 
that  factor,  it  is  selected  for  training.  All  the  factors 
can  then  be  considered,  with  respective  cutoff  scores 
indicating  the  most  important  tasks. 

For  example,  for  combat  tasks,  the  factor  of  Consequences  of 
Inadequate  Performance  may  be  important.  Thus  any  task  over 
6.0  (series)  would  be  selected.  Perhaps  the  commander  might 
also  feel  that  a  large  percentage  of  performers  indicates  a 
task  should  be  trained.  Thus,  only  tasks  with  over  50%  of 
the  incumbents  performing  would  be  selected  for  training. 
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Combining  these  two  cutoff  scores  would  mean  that  all  tasks 
performed  by  50%  of  the  soldiers  and/or  scoring  6.0  or 
higher  on  Consequences  of  Inadequate  Performance  would  be 
selected  for  training  (Note  that  the  establishment  of  the 
cutoff  point  is  somewhat  arbitrary). 

Alternatively,  a  method  can  be  devised  for  aggregating  a  set 
of  ratings  on  task  factors  to  form  a  summary  index  of 
training  priority  for  each  task. 

Tasks  are  then  rank  ordered  by  the  training  priority  index 
and  cutoff  values  are  established  indicating  the  number  of 
tasks  that  will  receive  initial  training  in  institutional 
courses.  Cutoff  values  can  be  developed  by  first  examining 
the  number  of  tasks  currently  assigned  to  each  training 
setting  in  the  MOS  (this  information  can  be  obtained  from 
the  Trainer's  Guide  for  the  MOS).  For  example,  the  Trainer's 
Guide  might  indicate  that  30  tasks  currently  receive  initial 
training  in  institutions.  The  cutoff  values  that  are 

currently  employed  in  the  MOS  must  then  be  examined  and 
updated  to  reflect  (1)  projected  resource  constraints  in 
each  training  setting,  and  (2)  major  changes  in  the  learning 
difficulty  of  the  tasks.  Initial  assignments  to  institu¬ 
tional  training  and  no  training  categories  are  made  by 
applying  the  cutoff  values  to  the  training  priority 
rankings. 

Table  3-3  contains  a  list  of  alternative  algorithms  for 
aggregating  task  factor  data.  Table  3-4  indicates  which 
algorithms  for  aggregating  task  factors  are  appropriate  for 
each  task  selection  model. 
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Table  3-3.  Potential  Algorithms  for 
Aggregating  Task  Factors 


Weighted  Z  Scores 

n 

TP I  =  Z  W  Z 
1=1  J 

where  TPI  is  the  training  priority  index;  W  is  the  weight  assigned  to 
the  i  the  task  factor;  and  Z^.  is  the  Z  score  of  the  1  task  on  the  jC 
factor;  and  N  is  the  number  oi  task  factors. 


Z .  .  =  X,  . 

_ hi 


SD. 


X. 

J 


where  X  is  the  raw  scor^ 
mean  of  the  X. .  for  the  i1 


of  the  i^  task  on 


the 


.th 


he  j  factor;  X  is  the 
the  standard  deviation  o 


factor;  and  SD  is 


the  X. .  for 


ij 


factor. 
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Individual  Task  Factor  Criteria 


In  this  method,  cut  points  are  established  for  individual  task  factors. 
Thus,  for  example,  a  cutoff  point  of  6.0  could  be  established  on  the 
factor,  consequences  of  inadequate  performance.  As  a  result,  any  task 
with  task  factor  scores  exceeding  6.0  would  be  selected  for  training. 
Similar  criteria  could  be  established  for  other  task  factors. 

e.g..  Select  for  training  if: 

>^6.0  on  Task  Factor  1 
>_  5.0  on  Task  Factor  2 
^5.0  on  Task  Factor  3 

Evaluate  each  task  against  criteria  for  each  task  factor. 


Task  Factor  Criteria  Algorithms 


A  task  factor  criteria  algorithm  specifies,  for  a  particular  combinatio 
of  values  on  task  factors,  whether  a  task  should  be  trained.  Such 
algorithms  are  not  really  used  to  aggregate  task  factor  scores  as  much 


vs‘ 


_ _ 1  . .  • _ _ 


Table  3-4.  Appropriate  Aggregation  Algorithms  for  Alternative 
Task  Selection  Models. 


Aggregation 

Algorithms 

Weighted 

7 

individual 

Task 

Factor 

Simple 

Task  Factor 
Criteria 

Task  Selection  Model^'"-*^^ 

Scores 

Criteria 

Weighting 

Algorithms 

3.1.2  Rate/Score  Tasks  on  Relevant  Characteristics 

OVERVIEW 

During  this  procedure,  methods  are  designed  to  collect  data 
on  the  task  factors  required  for  the  method  selected  in 
Procedure  3. 1.1.2  and  the  data  are  collected  and  compiled. 
An  overview  of  the  elements  comprising  this  procedure  is 
depicted  in  Figure  3-6. 

3. 1.2.1  Design  Methods  for  Rating/Scoring  Tasks 
PROCEDURE 

In  this  step,  methods  for  rating/scoring  tasks  are  devised, 
based  on  the  task  selection  method  being  used.  Procedures 
for  each  task  selection  method  are  described  below. 

o  Comparability  Analysis 

When  using  comparability  analysis  no  ratings/scores  are 
obtained  on  tasks.  The  user  can  proceed  to  Procedure  3.1.3. 

o  DIF  Algorithm 

An  example  of  a  data  collection  format  which  can  be  used  to 
employ  the  DIF  model  is  presented  in  Figure  3-7.  A  series 
of  three  questions  is  presented  to  job  incumbents/ 
supervisors  in  the  sequence  shown  in  the  figure. 

Of  course,  this  format  should  be  tailored  by  each  proponent 
school  to  meet  its  specific  needs. 
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NOTES:  1  TOTAL  NUMBER  OF  INCUMBENTS  AND  SUPERVISORS  MUST  BE 


DESIGNATED  BY  PROPONENT  SCHOOL. 

2  IF  POSSIBLE  DETERMINE  WHAT  WAS  DIFFICULT: 

LEARNING?  PERFORMANCE? 

3  THIS  SCALE  MUST  BE  DETERMINED  BY  PROPONENT  SCHOOL. 
A  NOTIONAL  CATEGORY  COULD  BE: 

a.  VERY  FREQUENT-NEARLY  EVERY  PROJECT. 

b.  MODERATE  FREQUENCY— +1  X  PER  WEEK. 

c.  INFREQUENT— +1  X  PER  MONTH. 


Figure  3-7  DIF  Data  Collection  Format 


o 


4-Factor  Model 


Data  must  be  obtained  on  the  four  factors  in  this  model 
either  from  the  Army  Occupational  Survey  Program  (AOSP)  of 
MILPERCEN  or  directly  from  surveys  administered  to  job 
incumbents/supervisors.  Each  of  the  four  factors  is 
described  in  detail  in  the  following  paragraphs. 

Percent  Performing 

The  criterion,  "percentage  of  job  incumbents  who  perform  the 
task",  indicates  those  tasks  that  are  most  often  performed 
on  the  job. 

For  example,  one  task  for  weather  technicians  is  "to  answer 
telephone  inquiries  about  the  weather."  If  you  found  that 
96  percent  of  all  weather  technicians  performed  this  task, 
the  implications  for  training  would  be  different  than  if  you 
found  that  only  10  percent  performed  it. 

In  the  above  example,  if  only  10  percent  of  job  incumbents 
perform  a  task,  there  is  a  strong  probability  that  90 
percent  of  your  training  resources  would  be  wasted  if  you 
trained  all  weather  technicians  to  perform  the  task.  In 
order  to  obtain  data  for  determining  the  percentage  of  job 
incumbents  performing  each  task,  simply  ask  "Do  you  perform 
this  task?"  or  derive  the  percentage  performing  from  answers 
to  other  related  questions.  If,  for  example,  in  collecting 
"frequency  of  task  performance”  data  one  of  the  possible 
responses  is  "never"  or  "do  not  perform",  you  will  have  the 
basic  data  for  determining  percent  performing. 


Task  delay  tolerance 

The  delay  tolerance  of  a  task  is  a  measure  of  how  much  delay 
can  be  tolerated  between  the  time  the  need  for  task  perfor¬ 
mance  becomes  evident  and  the  time  the  actual  performance 
begins.  There  are  some  tasks  normally  encountered  by  job 
incumbents  in  which  no  such  delays  can  be  tolerated.  The  job 
incumbent  who  encounters  the  task  must  be  capable  of  doing 
it  then  and  there,  without  needing  to  read  task  procedures 
or  finding  someone  to  advise  him  or  take  over  completely. 
For  other  tasks,  a  delay  of  a  few  minutes  or  perhaps  half  an 
hour  might  be  quite  acceptable,  or  even  mandatory,  while  the 
job  incumbent  gets  advice,  checks  technical  orders, 
regulations,  etc.  And  for  some  tasks,  there  might  be  time 
to  assemble  a  group  of  experts  to  confer  before 
proceeding.  The  delay  tolerance  of  a  task  is  a  measure  of 
how  much  delay  can  be  tolerated  between  the  time  the  need 
for  task  performance  becomes  evident  and  the  time  actual 
performance  begins.  The  following  are  examples  of  low  delay 
tolerance  tasks  requiring  immediate  pe "formance. 

EXAMPLES 

1.  Use  artificial  respiration  to  restore  the  breathing  of 
an  accident  victim. 

2.  Pull  ripcord  of  emergency  parachute  if  main  parachute 
fails. 

3.  Warn  suspect  of  his  legal  rights  before  questioning. 

4.  Film  historic  occasion  for  official  records. 

5.  Extinguish  fire  in  aircraft  engine  during  startup  on 
flight  line. 

Tasks  determined  to  have  a  low  delay  tolerance  should  be 
given  relatively  high  priority  for  selection  for  training. 
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Examples  of  tasks  having  a  higher  delay  tolerance,  thereby 
permitting  performance  delay,  would  include: 


EXAMPLES 

1.  Review  books  for  unit  library. 

2.  Refill  fire  extinguisher  after  use. 

3.  Advise  major  command  of  unit  manning  problem. 

4.  Fit  microphones  in  aircrew  oxygen  masks. 

A  high  delay  tolerance  does  not  exclude  a  task  from 
training,  but  indicates  that  other  factors  will  be  more  of  a 
basis  for  acceptance  or  rejection.  To  obtain  data  on  this 
criterion,  individuals  who  are  familiar  with  the  job  are 
asked  to  rate  the  amount  of  delay  that  can  be  tolerated 
before  task  performance  begins,  according  to  the  categories 
listed  below. 


(1) 

Extremely  low  -  performance  can 
indefinitely:  almost  never  urgent. 

be  put  off 

(2) 

Low . 

(3) 

Somewhat  below  average. 

(4) 

Average . 

(5) 

Somewhat  above  average. 

(6) 

High. 

(7) 

Extremely  high  -  task  performance 

instantly. 

must  begin 

Task  learning  difficulty 

The  learning  difficulty  of  a  task  refers  to  the  time, 
effort,  and  assistance  required  to  achieve  performance 
proficiency.  Some  tasks  encountered  in  each  specialty  are 
so  easy  or  so  familiar  that  they  can  be  readily  "picked  up" 
on  the  job  without  formal  training.  At  the  other  extreme, 


some  tasks  are  so  complicated  that  a  job  incumbent  can 
perform  them  adequately  only  after  a  sustained  period  of 
formal  training.  Other  tasks  lie  somewhere  between  these 
extremes  and  require  different  levels  of  training.  Tasks 
easy  enough  to  be  "picked  up"  on  the  job  without  training 
might  be: 

EXAMPLES 


(1)  Sweep  floors. 

(2)  Collect  food  trays  from  patients  in  hospital 
wards . 

(3)  Distribute  unclassified  correspondence  in  an 
of  f ice . 


Tasks  requiring  lengthy,  formal  training  might  be: 

EXAMPLES 

(1)  Diagnose  malfunction  in  an  airborne  radar  weapons 
system. 

(2)  Defuse  unexploded  enemy  bombs. 

(3)  Identify  parasites  in  clinical  specimens. 

To  obtain  data  on  the  factor  of  task  learning  difficulty, 
job  incumbents  or  supervisors  might  be  asked  to  rate  tasks 
as  to  the  training  time  required  to  achieve  proficiency,  or 
as  to  the  difficulty  of  "picking  up"  the  task  on  the  job 
without  formal  training.  The  following  rating  scale  might 
be  used: 

(1)  Extremely  low  -  extremely  easy  to  "pick  up" 
without  systematic  training. 


(3)  Somewhat  below  average. 

(4)  Average. 

(5)  Somewhat  above  average. 

(  6  )  High. 

(7)  Extremely  high  -  extremely  difficult  to  learn 
without  systematic  training. 

Consequences  of  inadequate  performance 

The  criterion  of  consequences  of  inadequate  performance 
points  to  the  need  for  selecting  tasks  for  training  that  are 
essential  to  job  performance,  when  needed,  even  if  they  are 
seldom  performed.  The  consequences  of  inadequate 

performance  on  certain  tasks  could  result  in  injury  to 
personnel,  loss  of  life,  or  damage  to  equipment.  Inadequate 
performance  could  have  a  serious  impact  on  the  mission,  the 
operation,  the  product,  the  equipment,  or  the  operator. 

EXAMPLES 


(1)  More  and  more  electronic  equipment  is  being 
transistorized  and  is  therefore  less  subject  to 
malfunction.  This  fact  should  reduce  the  amount 
of  maintenance  training  that  an  operator 
requires.  However,  there  may  be  a  number  of 
malfunctions  that,  although  occurring  very 
infrequently,  would  be  extremely  critical  if 
immediate  corrective  action  were  not  taken  by  the 
operator.  Severe  damage  to  the  equipment, 

materials,  or  products  might  result.  Under  such 
circumstances,  the  criticality  of  the  infrequently 
used  tasks  is  so  great  that  it  must  be  considered 
in  choosing  tasks  for  training. 


(2)  The  probable  consequences  of  inadequate 
performance  of  such  combat  tasks  as  "identify 
enemy  aircraft"  could  be  loss  of  life  and 
equipment. 

(3)  The  probable  consequence  of  inadequate  performance 

of  the  task  of  "write  trip  report"  is 

negligible.  If  this  task  were  selected  for 

training,  it  would  be  on  the  basis  of  factors 
other  than  probable  consequences  of  inadequate 
performance . 

To  obtain  data  on  this  criterion,  individuals  familiar  with 
the  job  are  asked  to  rate  probable  consequences  of 
inadequate  performance  of  each  task  according  to  the 

categories  listed  below. 

(1)  Extremely  low  -  if  performed  inadequately, 
consequences  are  negligible. 

(2)  Low. 

(3)  Somewhat  below  average. 

(4)  Average. 

(5)  Somewhat  above  average. 

(6)  High. 

(7)  Extremely  high-inadequate  performance  may  result 
in  mission  failure,  injury,  death,  or  damage  to 
important  equipment. 

Tables  3-5  and  3-6  contain  sample  sets  of  data  collection 
forms  for  the  four  factor  model. 
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Table  3-6. 

Sample  Data  Collection  Form  for  Four  Factor  Model 
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Table  3-6  {Continued) 


Consequences  of  Inadequate  Performance 


o 


8-Factor  Model 


Data  collection  for  the  8-Factor  Model  is  similar  to  that 
for  the  4-Factor  Model.  Those  factors  in  the  8-Factor  Model 
which  are  not  part  of  the  4-Factor  Model  are  described  in 
the  following  paragraphs. 

Percent  of  time  spent  performing 

The  percent  of  time  spent  performing  a  task  is  a  criterion 
suggesting  that  training  be  provided  to  assist  job 
incumbents  in  efficiently  performing  their  most  time 
consuming  tasks. 

EXAMPLE 

In  the  Protective  Equipment/Pressure  Suit  Specialty,  6.67 
percent  of  average  time  of  all  members  is  spent  performing 
the  task  of  "maintain  rigid  survival  kits".  Only  0,16 
percent  of  average  time  is  spent  on  the  task  of  "install 
microphones  in  oxygen  masks."  If  more  effective  training 
could  increase  productivity  50  percent  in  the  first  task, 
3.33  percent  of  the  total  time  of  all  members  might  be 
saved.  This  would  represent  a  significant  savings,  However, 
if  productivity  were  increased  50  percent  for  the  second 
task.  Only  0.08  percent  of  the  total  time  would  be  saved. 
This  represents  a  much  smaller  savings  opportunity. 

To  obtain  data  for  determining  the  percentage  of  time  spent 
performing  the  tasks  in  a  specially,  inputs  are  required 
from  a  large  number  of  job  incumbents.  Usually  they  are  not 
asked  to  state  the  percentage  of  their  time  spent  on  each 
task  because  such  a  question  would  be  very  difficult  to 


answer*  Instead,  they  are  usually  asked  to  rate  each  task 

as  to  the  amount  of  time  spent  performing  it  as  compared  to 
their  other  tasks. 

EXAMPLE 

You  are  to  rate  the  relative  amount  of  time  you  spend 

performing  each  task  in  your  present  job.  Relative  time 
spent  means  the  total  time  you  spend  doing  the  task  compared 
with  the  time  you  spend  on  each  of  the  other  tasks  in  your 
present  job.  The  following  scale  might  be  used  to  rate  the 
relative  amount  of  time  spent  on  each  task: 

1  very  much  below  average. 

2  below  average  time. 

3  for  slightly  below  average  time. 

4  for  about  average  time. 

5  for  slightly  above  average  time. 

6  for  above  average  time. 

7  for  very  much  above  average  time. 

From  the  above  information-  a  computer  program  such  as 
MILPERCEN's  Comprehensive  Occupational  Data  Analysis  Program 
(CODAP)  can  compute  and  print  out  the  average  percent  of 
time  spent  by  members  of  the  specialty  who  perform  the  task, 
and  the  average  percent  of  time  spent  by  all  members  of  the 
specialty. 

Frequency  of  Performance 

While  the  probable  consequences  of  inadequate  performance  of 
a  particular  task  are  serious  and  the  task  delay  tolerance 
in  low,  the  task  might  still  rate  low  for  training  priority 
if  it  is  rarely  performed. 


EXAMPLE 


For  a  medical  corpsman,  the  task  of  "deliver  baby"  is  so 
rarely  performed  that  it  probably  would  not  be  trained  in 
spite  of  the  serious  consequences  of  inadequate  performance 
and  the  relatively  low  task  delay  tolerance. 

On  the  other  hand,  if  a  task  is  performed  frequently,  the 
pay-off  in  terms  of  return  on  training  dollars  expended  is 
likely  to  be  great,  particularly  if  there  is  a  known  "best 
way"  to  perform  the  task.  A  practical  way  to  collect 
frequency  of  performance  data  on  tasks  is  to  rate  their 
frequency  of  performance  on  a  scale  such  as  the  following: 

(1)  Never  perform. 

(2)  Once  a  year. 

(3)  Once  every  three  months. 

(4)  Once  a  month. 

(5)  Once  a  week. 

(6)  Once  a  day. 

(7)  Several  times  a  day. 

Probability  of  deficient  performance 

The  criterion  of  probability  of  deficient  performance 
insures  that  training  is  given  in  those  essential  job  skills 
in  which  job  incumbents  frequently  perform  poorly.  In  any 

job,  some  tasks  are  more  difficult  to  accomplish  (or  easier 
to  bungle)  than  others.  By  tabulating  the  judgments  of 
knowledgeable  personnel  regarding  the  probability  of 
deficient  performance,  a  list  of  these  poorly  performed 

tasks  can  be  produced.  Training  of  these  tasks,  regardless 

of  their  criticality,  must  be  given  serious  consideration. 

EXAMPLES 

1.  If  equipment  downtime  is  often  caused  by  faulty 

soldering,  this  skill  may  require  additional 


emphasis  in  a  list  of  tasks  selected  for  training 
of  repairmen. 

2.  If  widespread  theft  of  items  guarded  by  military 
police  is  a  problem,  the  task  of  "guard  packages, 
materials,  and  property"  and  "prepare  physical 
security  plans"  may  require  additional  emphasis. 

To  obtain  data  on  the  criterion  of  probability  of  deficient 
performance,  supervisors  of  job  incumbents  might  be  asked  to 
rate  each  task  as  to  how  often,  according  to  the  scale 
below,  subordinates  in  the  MOS  perform  the  task  in  an 
unacceptable  manner: 

(1)  Very  much  below  average. 

(2)  Below  average. 

(3)  Slightly  below  average. 

(4)  Average. 

(5)  Slightly  above  average. 

(6)  Above  average. 

(7)  Very  much  above  average. 

Immediacy  of  performance  or  time  between  job  entry  and  task 


performance 

The  criterion  of  the  time  interval  between  completion  of 
training  and  initial  performance  of  the  task  on  the  job  has 
some  significance  in  selecting  tasks  for  training.  Here,  the 
determining  factors  are: 


Whether  or  not  there  is  a  high  probability  of  a 
graduate  encountering  the  task  on  the  job  fairly 
soon  after  completing  training.  "Fairly  soon" 
means,  in  this  context,  that  task  encountered 
within  the  first  year  after  training  would. 


everything  else  being  equal,  be  weighted  more 
heavily  for  selection  t-han  those  not  encountered 
until  one  to  two  years  later  and 

(2)  The  predicted  or  measured  amount  of  decay  of  the 
skill  that  will  take  place  during  the  time 
interval. 

EXAMPLE 

The  ability  to  send  and  receive  Morse  Code  is  a  relatively 
difficult  skill  to  acquire.  If  the  skill  is  not  used,  a 
considerable  amount  of  decay  is  certain  to  occur.  If  the 
skill  is  only  rarely  needed  by  personnel,  it  may  be  wise  to 
exclude  the  task  in  the  training  given  to  all  trainees. 
However,  if  the  skill  is  likely  to  be  used  immediately  after 
graduation  by  most  graduates,  it  probably  should  be  included 
in  the  training  for  all  trainees. 

To  obtain  data  on  this  criterion,  job  incumbents  and  others 
might  be  asked  to  rate  the  time  between  job  entry  and  task 
performance  on  a  scale  such  as  that  listed  below. 


(1) 

Task  not  yet  performed. 

(2) 

Task  first  performed  more 

ass ignment . 

than  4  years 

after 

(3) 

Task  first  performed  between 

ass ignment , 

2 

and  4  years 

after 

(4) 

Task  first  performed  between 
ass ignment . 

1 

and  2  years 

after 

(5) 

Task  first  performed  between 
after  assignment. 

6 

months  and  1 

year 

(6)  Task  first  performed  between  3  months  and  6  months 
after  assignment. 

(7)  Task  performed  during  first  3  months  of 

ass ignment . 

Sample  data  collection  forms  for  those  factors  not  included 
in  the  4-Factor  Model  are  presented  in  Table  3-7. 

3. 1.2. 2  Rate/Score  Tasks 

During  this  procedure,  data  collection  takes  place  using  a 
minimum  of  10  to  20  raters  for  each  task.  Additional  raters 
may  be  required  if  the  total  number  of  judgments  required  of 
each  rater  becomes  excessive.  The  number  of  judgments 
required  can  be  computed  by  multiplying  the  number  of  task 
factors  by  tasks.  Raters  may  experience  fatigue  after 
making  several  hundred  judgments  and  the  reliability  of 
subsequent  ratings  may  be  unsatisfactory. 

3.1.3  Apply  Algorithm/Aggregation  Method 

OVERVIEW 

During  this  procedure,  data  collected  during  Procedure  3.1.2 
will  be  compiled  and  aggregated,  task  selection  criterion/ 
cutoff  scores  will  be  developed,  and  tentative  task 
selections  will  be  made.  An  overview  of  this  procedure  is 
presented  in  Figure  3-8. 

PROCEDURE 

3.  1.3.1  Apply  Aggregation  Method 

In  this  step,  data  will  be  compiled  and  then  aggregated 
using  one  of  the  methods  depicted  in  Table  3-3.  The 
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Sample  Data  Collection  Form  for  Eight  Factor  Model 
Task  Selection  Survey  for  MOS  31M 
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Figure  3-8.  Apply  Algorithm/Aggregation  Method  (3.1.3). 
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worksheets  provided  in  Table  3-8  can  be  used 
data  obtained  in  Procedure  3.1.2.  These  data 
of  mean  ratings/scores  on  each  task  factor 
Thus,  each  entry  to  one  of  the  worksheets  in 
consist  of  the  average  rating/score  obtained 
one  task  factor  (sum  across  all  raters  and 
number  of  raters). 


to  compile  the 
should  consist 
for  each  task. 
Table  3-8  will 
by  one  task  on 
divide  by  the 


Secondly,  an  aggregation  method  (if  appropriate)  will  be 
applied  to  the  data  contained  in  the  worksheets,  resulting 
in  a  summary  score/rating  for  each  task.  Summaries  of  each 
of  the  data  aggregation  methods  are  contained  in  Table  3-3. 

3. 1.3. 2  Develop  Task  Selection  Criterion/Cutoff  Score(s) 

Criterion/cutoff  scores  are  identified  in  this  procedure 
that  will  be  used  to  determine  which  tasks  will  be  selected 
for  training.  These  cr iter ion/cutof f  scores  can  be 
determined  by: 


(1)  Using  the  number  of  tasks  currently  trained  for  a 

similar  system.  After  determining  the  number  of 
tasks  on  the  similar  system  a  cutoff  point  on  the 
summary  measure  is  set  which  selects  the 

appropriate  number  of  tasks  for  training. 

(2)  Examining  available  training  resources  to  deter¬ 
mine  the  available  training  time  in  various 
settings.  This  examination  may  reveal  a  limit  on 
the  number  of  tasks  which  can  be  selected  for 
training.  As  a  result,  a  cutoff  point  can  be  set 
which  will  produce  the  appropriate  number  ot 
tasks. 
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Table  3-8  Data  Summary  Worksheets 


SYSTEM 


Comparability  Analysis  (Comp) 


(a)  NEW  TASK  TITLE 


(b)  COMPARABLE  TASK 


(c)  COMPARABLE  TASK 
TRAIN/NO  TRAIN 


PERCENT 


Table  3-8  Data  Summary  Worksheets  DIF  Algorithm(Continued) 


SYSTEM 


low, 


(3)  Using  objective  indices  of  criticality  of  training 
such  as  individual  task  factor  criteria  or  task 
factor  criteria  algorithms  (See  Table  3-3). 

(4)  Using  some  combination  of  the  above  methods. 

3. 1,3. 3  Select  Tasks  for  Training 

The  criterion/cutoff  scores  determined  in  procedure  3.  1.3. 2 
are  applied  in  this  procedure  to  produce  a  tentative  list  of 
tasks  selected  for  training.  The  worksheet  presented  in 
Table  3-9  can  be  used  to  assist  in  this  process. 

3.1.4  Review  Current  Task  Selection 

During  this  procedure,  tentative  task  selections  for 
training  are  reviewed,  revised  if  necessary,  and  finalized. 
An  overview  of  this  procedure  is  presented  in  Figure  3-9. 

3.  1.4.1  Review  Tentative  Task  Selection 

The  purpose  of  this  step  is  to  review  the  tentative  task 
selection  accomplished  in  procedure  3. 1.3. 3.  If  the 

tentative  task  selection  is  adequate,  the  user  is  finished. 
If  the  number  of  tasks  selected  for  training  either  exceeds 
or  falls  short  of  the  planned  training  load,  adjustments 
will  need  to  be  made  to  the  task  selection  criterion/cutoff 
scores . 

3. 1.4. 2  Revise  Criterion/Cutoff  Scores 

In  this  step,  revised  criterion/cutoff  scores  are  developed 
so  that  the  task  selection  list  will  match  available 
training  resources. 


e  3-9.  Worksheet  for  Selecting  Tasks  for  Training  (TSEL) 


3. 1.4.3  Apply  Revised  Criterion/Cutoff  Scores 


Revised  criterion/cutoff  scores  are  applied  in  this  step  to 
select  tasks  for  training.  The  steps  described  in  procedure 
3.  1.3. 3  are  repeated  and  a  new  list  of  tasks  selected  for 
training  is  developed. 

3.1.5  Document  Data 

The  train-no  train  decision  should  be  documented  in  the  SDT 
after  tasks  have  been  assigned  to  training  settings  in 
Procedure  3.2. 

3.2  ASSIGN  TASKS  TO  TRAINING  SETTINGS 

Once  tasks  have  been  selected  for  training,  they  should  be 
further  analyzed  to  determine  the  most  appropriate  training 
setting.  The  key  decision  is  to  allocate  tasks  between 
formal  school  settings,  (for  example,  advanced  individual 
training)  and  unit  training  settings  (for  example, 
supervised  on-the-job  training).  Once  this  decision  is 
made,  assignments  within  these  two  general  types  of  settings 
can  be  made. 

This  procedure  describes  two  basic  approaches  to  assigning 
tasks  to  training  settings.  One  approach  is  comparability 
analysis  which  bases  training  setting  assignments  on  assign¬ 
ments  made  to  comparable  tasks.  The  second  approach  involves 
selecting  criteria  for  making  training  setting  assignments, 
collecting  data,  applying  an  aggregation  algorithm  and 
making  training  setting  assignments  based  on  the  aggregated 
criterion  scores.  An  overview  diagram  depicting  the  steps 
in  this  procedure  is  presented  in  Figure  3-10. 
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Figure  3-10.  Overview  of  Assignment  Tasks  to  Training  Settings  (3.2). 

3-56 

.  ,«■  A *  '  *•  ‘  v-  V  .•  *  •  ’*-  •  \* 

■  t'.  -V  rfL.  Ld  J<  W  :  Jl  '-Jif-m  'J  •«.  -.m  \  *  - 


3.2.1  Select  Setting  Assignment  Method 


There  are  two  basic  approaches  to  assigning  tasks  to 
training  settings:  comparability  analysis  and  the  use  of  a 
selection  algorithm  (X-factor  method). 

o  Comparability  Analysis 

Comparability  analysis  may  be  used  to  provide  crude  training 
setting  assignments  during  the  early  phases  of  the  acquisi¬ 
tion  process.  The  first  step  in  the  application  of  compar¬ 
ability  analysis  is  to  identify  a  comparable  existing  task 
for  each  of  the  tasks  in  the  New  System  (if  comparability 
analysis  was  used  in  Procedure  3.1,  the  results  will  support 
this  analysis).  A  comparable  existing  task  may  be  defined 
as  a  task  which  (1)  is  performed  on  comparable  equipment  or 
(2)  involves  the  same  type  of  task  action  (that  is,  the 
action  verb  for  the  comparable  task  matches  or  is  highly 
similar  to  the  action  verb  for  the  New  system  task). 

The  second  step  in  comparability  analysis  is  to  assign  the 
New  system  task  to  the  same  training  setting  (institutional 
training,  unit  training)  assigned  to  their  comparable  tasks. 

o  X-Factor  Approach 

In  this  approach,  criteria  are  identified  which  are  relevant 
to  the  training  setting  assignment  decision.  These  criteria 
will  typically  include  the  same  criteria  used  in  the 
procedure  for  selecting  tasks  for  training.  The  six  factors 
most  likely  to  be  used  are  task  delay  tolerance,  task 
learning  difficulty,  percent  time  performing,  percent 
performing,  time  between  initial  MOS  training  and  required 


job  performance,  and  consequences  of  inadequate  performance. 
Data  is  collected  (If  4-Factor  or  8-Factor  models  were  used 
in  Procedure  3.1,  much  of  the  required  data  may  already  be 
available)  directly  by  the  analyst  or  from  surveys 
administered  by  the  Army  Occupational  Survey  Program  (AOSP) 
of  MILPERCEN.  Once  information  has  been  collected  for  each 
of  the  X-factors  for  every  task  during  Procedure  3.1,  there 
will  be  a  large  data  base  to  use  for  assigning  tasks  to 
training  settings.  The  user  should  select  the  same  subset 
of  factors  used  in  Procedure  3.1. 

The  X-Factor  approach  is  more  comprehensive  than  the 
comparability  analysis  approach  and  has  greater  data 
requirements.  Therefore,  the  X-Factor  approach  and  be  more 
appropriate  later  in  the  acquisition  process  when  more 
accurate  training  setting  assignments  are  required. 

3. 2. 2.1  Identify  Training  Settings  for  Comparable  Tasks 

If  the  comparability  analysis  approach  is  selected,  then 
training  settings  for  comparable  tasks  are  identified  (if 
comparability  analysis  was  employed  in  Procedure  3.1,  this 
data  may  already  be  available).  These  existing  training 
setting  assignments  are  then  simply  transfer  :  ed  to  the  New 
system  tasks.  Once  the  assignments  have  bee  i  made,  the  user 
can  advance  to  Procedure  3.2.3  (Document  Data). 

3. 2. 2. 2  Obtain  Task  Data 

If  the  user  selects  the  X-Factor  approach,  data  must  be 
collected  on  the  task  factors.  First,  however,  the  user 
must  select  the  criteria  to  be  included  in  the  analysis. 
Two  of  the  more  useful  criteria  for  the  training  assignment 
decision  are: 


o 


Percent  performing  -  The  greater  the  proportion  of 
individuals  in  an  MOS  who  perform  a  task  on  the 
job,  the  greater  the  payoff  for  training  them  in 
an  institution.  Thus,  if  a  specific  duty  position 
accounts  for  a  substantial  percentage  of  the 
personnel  within  an  MOS,  then  that  duty  position 
is  identified  to  be  trained  in  the  institution. 
For  example,  for  MOS  95B,  Military  Police,  at 
Skill  Level  1,  there  are  several  different  duty 
positions,  such  as  military  police,  desk  clerk, 
radio  dispatch  clerk,  fingerprint  clerk,  machine- 
gunner,  investigator,  prisoner-of-war  processing 
specialist,  etc.  The  majority  of  these  duty 
positions  account  for  a  relatively  small  percent 
of  the  total  MOS  strength.  However,  the  training 
assignment  decision  is  easy  because  the  position 
of  military  police  accounts  for  approximately  60 
percent  of  the  MOS  for  Skill  Level  1. 

o  Time  between  training  and  required  job  performance 
-  If  a  task  is  complex  and  requires  practice  to 
maintain  proficiency  then  tasks  which  are  not 
performed  by  soldiers  within  6  months  after 
training  should  be  trained  in  the  unit. 

o  Consequences  of  Inadequate  Performance  -  This 
criteria  is  important  insofar  as  it  indicates 
which  tasks  may  need  concentrated  or  professional 
instruction  that  can  best  be  accomplished  in  an 
institutional  setting. 

Data  on  these  or  other  selected  criteria  can  either  be 
collected  in  surveys  conducted  directly  by  the  analyst  or 


through  the  Army  Occupational  Survey  Program  (AOSP)  (more 
details  on  procedures  for  collecting  this  data  are  provided 
in  Section  3.1.)  These  data  can  be  summarized  on  a 
worksheet  like  that  shown  in  Table  3-10.  Data  entries  to 
this  worksheet  should  consist  of  mean  scores/ratings  for 
each  task  on  each  criteria. 

As  part  of  the  data  collection  process,  the  user  should  also 
examine  the  Trainer's  Guide  for  the  MOS.  This  guide  will 
define  the  specific  training  settings  currently  used  for  the 
MOS.  If  a  new  MOS  is  involved,  training  settings  for  other 
comparable  MOSs  should  be  used  (see  Section  2.0  for 
information  on  identifying  comparable  MOSs). 

3. 2. 2. 3  Make  Training  Setting  Assignments 

Training  setting  assignments  must  be  made  in  a  two  step 
process.  First,  tasks  should  be  assigned  to  the  either 
institutional  or  unit  training  setting  categories.  Second, 
specific  settings  within  each  of  these  general  categories 
must  be  determined.  It  should  be  noted  that  within  the 
institutional  training  setting  category,  there  is  a  close 
correspondence  between  specific  training  settings  and  skill 
level.  Thus,  if  you  know  the  skill  level  of  a  task  and  have 
made  the  assignment  to  the  institutional  training  category 
the  assignment  of  tasks  to  specific  institutional  training 
settings  is  largely  determined.  Listed  below  are  the 
relationships  between  skill  level  and  institutional  training 
settings. 

o  Skill  Level  1  -  Basic  Training,  Advanced 

Individual  Training  (AIT),  or  One  Station  Unit 
Tra in ing  (OSUT ) . 
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o  Skill  Level  2  -  AIT,  OSUT,  or  Primary  Leadership 

Course . 

o  Skill  Level  3  -  Basic  Technical  Course. 

o  Skill  Level  4  -  Advanced  Noncommissioned  Officer 

Course . 

o  Skill  Level  5  -  Senior  Noncommissioned  Officer 

Course. 

Some  of  the  specific  training  typically  included  under  the 
unit  training  category  are  Supervised  On-the-Job  Training 
(SOJT),  Self-Study,  and  Scheduled  Training  (SCD).  These 
settings  do  not  vary  with  skill  level. 

Training  may  be  prescribed  for  a  task  in  both  institutional 
and  unit  settings.  In  these  cases,  a  distinction  is  made 
between  initial  training  to  proficiency  (called 
qualification  training)  and  subsequent  refresher  training 
(called  additional  training).  This  procedure  is  primarily 
concerned  with  the  identification  of  settings  of  initial 
train  .ng  assignments  since  most  additional  training  takes 
place  in  units. 

The  method  employed  to  make  initial  setting  assignments  is 
presented  in  Figure  3-11.  Note  that  this  method  is  very 
similar  to  the  method  used  to  select  tasks  for  training. 

3. 2. 2. 3.1  Select  and  Apply  Aggregation  Method 

In  this  procedure,  data  contained  in  Table  3-10  will  be 
aggregated  using  one  of  the  methods  depicted  in  Table  3- 


ii.  inis  will  result  in  a  summary  score/rating  tor  each 
task.  More  details  in  applying  these  aggregation  methods  are 
provided  in  Procedure  3.1. 


3.2. 2.3.2  Determine  and  Apply  Setting  Assignment  Criterion/ 
Cutoff  Scores 


Criterion/cutoff  scores  are  identified  in  this  procedure 
that  will  be  used  to  assign  tasks  to  training  settings. 
These  criterion/cutoff  scores  must  take  into  consideration 
any  constraints  on  training  resources  (e.g.,  time  available 
in  AIT  for  training,  availability  of  equipment  at  an 
institution).  The  criterion/cutoff  scores  can  be  determined 
by : 

o  Using  the  number  of  tasks  currently  trained  in  a 
setting  for  a  similar  system.  After  determining 
the  number  of  tasks  trained  in  a  setting  on  the 
similar  system,  a  cutoff  point  on  the  summary 
measure  is  set  which  selects  the  same  number  of 
tasks . 

o  Examining  available  training  resources  to 
determine  the  available  training  time  in  each 
setting.  A  cutoff  score  can  then  be  set  which 
will  produce  the  appropriate  number  of  tasks. 

o  Using  objective  indices  of  appropriateness  of  a 
setting  for  a  particular  task  such  as  individual 
task  factor  criteria  or  task  factor  criteria 
algorithms  (S me  Table  3-11). 


o 


Using  some  combination  of  the  above  method 


Once  tasks  have  been  assigned  to  the  institutional  training 
category,  they  should  be  assigned  to  specific  settings 
within  each  category.  As  indicated  in  Section  3. 2. 2. 3,  this 
is  largely  determined  by  the  skill  level  of  the  task. 
Additional  guidance  can  be  provided  by  examining  the 
assignments  made  to  similar  tasks. 

The  resulting  task  assignments  will  determine  the  location 
of  qualification  training,  but  in  most  instances  additional 
t  aining  will  be  required.  For  example,  if  AIT  is  the 
setting  for  qualification  training,  SOJT  would  likely  to  be 
selected  for  additional  training.  This  procedure  does  not 
specifically  address  the  question  of  training  sites  for  such 
additional  training  since  most  of  this  training  takes  place 
in  the  unit. 

3.2.3  Document  Data 

In  this  procedure,  tasks  which  are  selected  for  training  and 
assigned  to  training  settings  are  entered  into  the  SDT  as 
shown  in  Table  3-12. 

3.3  IDENTIFY  SKILLS  AND  KNOWLEDGES 

OVERVIEW 

During  this  step,  skills  and  knowledges  are  determined  for 
the  BCS  and  New  system  tasks  which  were  selected  for 
training  in  Procedure  3.2. 

PROCEDURE 

An  overview  of  this  procedure  is  presented  in  Figure  3-12. 
A  worksheet  which  can  be  used  to  support  this  procedure  is 
presented  in  Table  3-13. 


Table  3-11.  Aggregation  Methods  for 

Assigning  Tasks  to  Training  Settings 


Weighted  2  Scores 
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where  TPI  is  the  training  priority  index;  W  is  the  weight  assigned  to 
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task  on  the  j  factor;  X.  is  the 
factor;  and  SD.  Is  the  standard  deviation  of 


Individual  Task  Factor  Criteria 


In  this  method,  cut  points  are  established  for  individual  task  factors. 
Thus,  for  example,  a  cutoff  point  of  6.0  could  be  established  on  the 
factor,  consequences  of  inadequate  performance.  As  a  result,  any  task 
with  task  factor  scores  exceeding  6.0  would  be  selected  for  training. 
Similar  criteria  could  be  established  for  other  task  factors. 


e.g..  Select  for  training  if: 
>_  6.0  on  Task  Factor  1 
>^5.0  on  Task  Factor  2 
>  5.0  on  Task  Factor  3 


Evaluate  each  task  against  criteria  for  each  task  factor. 


Tesk  Factor  Criteria  Algorithms 

A  task  factor  criteria  algorithm  specifies,  for  a  particular  combination 
of  values  on  task  factors,  whether  a  task  should  be  trained.  Such 
algorithms  are  not  really  used  to  aggregate  task  factor  scores  as  much 
as  to  lead  the  analyst  to  a  solution. 


Table  3-13  Skill  and  Knowledge  Worksheet  (Skill) 


SYSTEM 


MOS/3KILL  LEVEL 


(a)  TASK: _ 

(b)  COMPARABLE  TASK: 


COMPARABLE/UNMODIFIED  TASK  SKILLS 

(c)  SKILL  (d)  SOURCE 


MODI  FI  ED/NEW  TASK  SKILLS 
(e)  SKILL  (f)  SOURCE 


To  begin  the  procedure,  you  must  examine  the  BCS  and  New 
system  task  list.  These  tasks  must  be  then  categorized  into 
two  groups,  (1)  added  or  modified  tasks  and  (2)  unmodified 
existing  tasks.  For  unmodified  tasks,  existing  skill  and 
knowledges  can  be  utilized.  Table  3-14  lists  some  of  the 
data  sources  which  can  be  used  to  obtain  information  or 
skills  and  knowledges  for  existing  tasks.  As  skill  and 
knowledge  information  on  existing  tasks  is  obtained,  it 
should  be  entered  into  columns  (a)  and  (b)  of  the  Skill  and 
Knowledge  (SKILL)  Worksheet. 


To  estimate  skills  and  knowledges  for  added  or  modified 
tasks,  comparability  analysis  techniques  must  be  used.  More 
specifically,  a  comparable  existing  task  must  be  identified 
and  information  on  the  skills  and  knowledges  of  this  task 
must  be  collected.  Skills  and  knowledges  from  the 
comparable  task  must  then  be  added  or  deleted  to  reflect  the 
differences  between  the  equipments  and  functions  associated 
with  the  comparable  task  and  the  added  or  modified  task 
(Information  on  equipments  and  functions  should  be  available 
from  Procedure  1.0) 

A  comparable  existing  task  may  be  defined  as  a  task  which 
(1)  is  performed  on  comparable  equipment  (procedures  for 
identifying  comparable  equipments  are  listed  under  Function 
1.0)  and  (2)  involves  the  same  type  of  task  action  (that  is, 
the  action  verb  for  the  comparable  task  matches  or  is  highly 
similar  to  the  action  verb  for  the  New  System  task). 


In  addition,  if  it  is  possible,  the  comparable  task  should 
be  selected  from  the  same  occupational  specialty  (MOS)  as 
the  New  system  task. 


Table  3-14.  Skill  and  Knowledge  Data  Sources 

Soldier  Manuals** 

Trainer's  Guide** 

Job  Book** 

CODAP** 

LSAR* 

ARTEPs** 

Field  Manuals** 

Maintenance  Manuals** 

Skill  Performance  Aids** 

Maintenance  Allocation  Charts* 

Training  Development  Information  System  (TD1S)** 
Contractor  Supplied  Training  Programs* 

Program  of  Instruction  (POI) 


All  skill  and  knowledge  statements  should  be  constructed  in 
accordance  with  existing  ISD  guidelines.  (See  Schulz  and 
Farrell,  1980  and  TRADOC  Pam  350-30  for  a  description  of 
these  guidelines). 

Once  the  skills  and  knowledges  for  the  new  or  modified  tasks 
have  been  identified,  they  should  be  entered  into  column  (e) 
and  (f)  of  the  SKILL  worksheet. 

Once  the  SKILL  worksheets  have  been  completed,  relevant  data 
from  these  worksheets  should  be  entered  into  the  SDT.  Table 
3-15  describes  the  most  desirable  order  for  entering  data 
into  the  SDT.  Additional  guidance  for  entering  data  into 
the  SDT  is  provided  in  the  SDT  User's  Guide. 

During  the  later  phases  of  the  acquisition  process  you  may 
want  to  do  a  more  detailed  task  analysis  and  enter  some 
additional  data  on  tasks  in  the  SDT.  Some  of  the  types  of 
data  typically  collected  during  these  types  of  more  detailed 
task  analyses  are: 

o  Task  Elements 

o  Preceding  and  Succeeding  Tasks  (in  terms  of 
mission  performance) 
o  Tools/Test  Equipment 

o  Failure  Modes  (maintenance  tasks  only) 
o  Initiating  Cues 

o  Terminating  Cues 

o  Learning  Objectives 

o  Performance  Measures 


o 


Task  Frequency  and  Duration  Measures 
o  Work  Area/Task  Area 

o  Amount  of  Supervision 

o  Number  of  Performing 

If  necessary,  these  items  can  be  generated  by  the  same 
comparability  analysis  techniques  used  to  estimate  skills 
and  knowledges.  The  sequence  for  entering  this  data  into 
the  SDT  is  displayed  in  Table  3-15. 

3.4  DEVELOP  TARGET  POPULATION  DESCRIPTION 

OVERVIEW 

Target  population  descriptions  must  be  developed  for  each 
job  within  an  MOS  for  the  New  system.  A  job  is  a  set  of 
related  duty  positions  within  an  MOS  (see  TRADOC  Pam  351-4, 
Job  and  Task  Analysis  Handbook,  for  a  more  detailed  descrip¬ 
tion  of  what  constitutes  a  job)  .  The  elements  of  a  job- 
description  for  a  new  systems,  per  TRADOC  Pam  351-4,  are  (a) 
skill  level,  (b)  occupational  specialty,  (c)  educational 
level,  (d)  prerequisite  training,  prerequisite  education, 
(including  reading  grade  level),  (e)  performance  data  (if 
available),  (f)  enlistment  rates,  and  (g)  overseas  tour 
length. 

PROCEDURE 

Table  3-15  describes  how  each  element  of  the  target 
population  description  can  be  developed  during  the  early 
phases  of  the  acquisition  process.  Skill  level  and 
occupational  specialty  are  determined  during  Procedure 
2.0.  For  the  remaining  elements  of  the  target  population 


Number  Performing  Tasks  -  -  2  (optional) 


Table  3-16.  Procedures  for  Developing 

Target  Population  Description 
Elements 


ELEMENT 

METHOD  FOR  DEV  ‘'PING 

Skill  Level 

Obtain  from  P  cedure  2.0 

Occupational  Specialty 

Obtain  from  P.  ^dure  2.0 

Educational  Level 

Use  Data  From  Existing  or 
Comparable  MOS 

Prerequisite  Training 

Use  Data  From  Existing  or 
Comparable  MOS 

i 


Performance  Data 


Use  Data  From  Existing  or 
Comparable  MOS 


Enlistment  Rates 


Use  Data  From  Existing  or 
Comparable  MOS 


Overseas  Tour  Length 


Use  Data  From  Existing  or 
Comparable  MOS 


description,  current  data  for  the  occupational  specialty 
should  be  employed.  If  a  new  occupational  specialty  is 
involved,  data  from  a  comparable  MOS  should  be  employed. 1 
Procedures  for  identifying  comparable  MOSs  are  listed  under 
Procedure  2.0. 

A  worksheet  for  documenting  the  target  population 

description  is  provided  in  Table  3-17. 

3.5  SEQUENCE  TASKS/SKILLS  FOR  TRAINING 

OVERVIEW 

During  this  procedure,  the  tasks/skills  associated  with  each 
setting  are  placed  in  the  sequence  in  which  they  must  be 
trained . 

PROCEDURE 


An  overview  of  the  procedure  for  sequencing  tasks  for 
training  is  provided  in  Figure  3-13.  First  the  tasks 

associated  with  each  setting  are  identified.  This 

information  will  be  available  from  Procedure  3.2. 


This 


Once  tasks  for  each  setting  have  been  identified,  these 
tasks  must  be  sorted  into  groups  of  "functionally  similar" 


1  This  procedure  only  provides  gross  estimates  of  the 
target  population  description  elements  that  are  appropriate 
for  the  earliest  phases  of  the  acquisition  process.  More 
specifically,  the  procedure  does  not  attempt  to  adjust  the 
target  population  elements  to  account  for  task  modifications 
or  additions  within  an  MOS. 


Table  3-17.  Target  Population  Worksheet  (POP) 


Figure  3-13.  Overview  Diagram:  Sequence  Tasks/Skills  for  Training  (3.5) 


tasks.  There  are  two  criteria  for  identifying  functionally 
similar  groups  of  tasks. 

(1)  Action  Verb  Similarity  -  Tasks  should  be  sorted 
into  groups  which  have  similar  task  action  verbs. 

(2)  Equipment  Similarity  -  Tasks  should  be  sorted  into 
groups  which  involve  comparable  or  similar  equip¬ 
ments.  Procedures  for  identifying  comparable 
existing  equipments  are  provided  under  Procedure 
1.0. 

Note  that  the  criteria  for  identifying  functionally  similar 
tasks  is  the  same  as  the  criteria  for  identifying  comparable 
existing  tasks,  as  described  in  Procedure  3.3. 

Once  tasks  have  been  sorted  into  functional  groups,  the 
groups  must  be  sequenced  to  reflect  the  learning  dependency 
relationships  among  the  groups  (that  is  what  groups  of  tasks 
represent  prerequisite  skills  and  knowledges  which  must  be 
learned  before  other  groups  of  tasks).  Detailed  guidance 
for  identifying  these  types  of  learning  dependencies  is 
provided  in  Schulz  and  Farrell  (1980). 

Once  the  groups  have  been  sequenced  to  reflect  learning 
dependencies,  they  must  be  rank-ordered  to  reflect  the 
estimated  learning  difficulty  of  the  tasks  within  the  groups 
(Guidelines  for  estimating  learning  difficulty  were 
presented  in  Procedure  3.1). 

After  the  functional  groups  have  been  sequenced,  the  tasks 
within  each  group  should  be  sequenced  using  the  same 
criteria  to  sequence  the  groups  (i.e.,  learning  dependency 
and  learning  difficulty). 


A  worksheet  which  can  be  used  to  document  the  results  of 
this  procedure  is  presented  in  Table  3-18. 

3.6  CONSTRUCT  QUASI-PROGRAM  OF  INSTRUCTIONS 

During  this  procedure,  quasi-program  of  instructions  (QPOIs) 
are  developed  for  individual  institutional  training  courses. 
The  quasi-POI  provides  a  convenient  vehicle  for  summarizing 
the  content  of  a  course/instructional  program  during  the 
early  phases  of  development.  It  provides  the  information 
needed  to  determine  the  resource  and  cost  requirements  of  a 
course/program,  to  estimate  its  efficiency/effectiveness, 
and  to  plan  for  its  development.  A  worksheet  which  can  be 
used  to  support  the  procedure  is  provided  in  Table  3-19. 
This  worksheet  summarizes  the  essential  elements  of  the 
QPOI.  It  should  be  noted  that  ETES  currently  contains 
procedures  for  developing  QPOIs  for  individual  institutional 
training  courses. 

ETES  does  not  currently  include  procedures  for  developing 
QPOIs  for  individual  unit  training  or  collective  training  or 
for  developing  outlines  for  reference  material. 

PROCEDURE 

An  overview  of  the  procedure  for  constructing  quasi-POIs  for 
instructional  training  is  provided  in  Figure  3-14. 

There  are  five  steps  in  the  construction  of  quasi-POIs  for 
individual  institutional  training  courses:  (1)  identifica¬ 
tion  of  courses  and  course  modules;  (2)  identification  of 
the  methods  to  be  used  within  each  course  module;  (3) 
identification  of  the  contact  hours  to  be  devoted  to  each 
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Table  3-19.  QPOI  Worksheet  -  Part  1  (QPOM) 


method  within  a  module;  (4)  determination  of  course  length, 
and  (5)  documentation  and  formatting  of  the  QPOI. 


More  details  on  each  of  these  steps  is  provided  'in  the 
sections  which  follow. 

3.6.1  Identify  Courses  and  Course  Modules 

The  training  task  sequence  relationships  which  were 
identified  in  Procedure  3.5  for  all  the  tasks  within  a 
setting  are  examined  and  logical  break-points  in  the 
sequence  are  identified  where  tasks  may  be  grouped  together 
to  form  courses  and  course  modules.  Guidance  for  the 

construction  of  courses  and  modules  is  provided  by  the 
examination  of  the  types  of  courses  currently  used  in  the 
MOS  and/or  predecessor  system  and  the  modules  within  these 
courses  which  deal  with  similar  tasks.  Similar  tasks  may  be 
identified  by  examining  (e)  the  task  action  verb  and  (b)  the 
equipment  on  which  the  task  operates.  (Procedures  for 
identifying  "similar"  tasks  were  identified  in  Procedure 
3.3.  Identification  of  comparable  modules  can  be 

facilitated  by  identifying  the  comparable  existing  course 
which  most  closely  matches  the  task  requirements  of  the  new 
system.  (Usually  this  comparable  course  is  the  course  is 
the  course  from  the  predecessor  system). 

Modules  may  be  added  to  or  deleted  from  the  comparable 
course  to  reflect  the  New  system  task  requirements.  An 

example  of  a  worksheet  which  can  be  used  to  document 

modifications  to  an  existing  course  is  presented  in  Table  3- 
20.  Data  sources  for  identifying  comparable  courses  are 
listed  in  Appendix  E. 


TABLE  3-20.  SAMPLE  COURSE  MODIFICATION  WORKSHEET  (CMOD) 


As  part  of  the  module  identification  process,  the  user  must 
identify  non-academic  modules.  Non-academic  modules  are 
modules  which  are  not  associated  with  task  requirements. 
For  example,  testing  modules  are  usually  included  at  the  end 
of  each  major  section  of  a  course.  In  addition,  modules  are 
often  added  to  a  course  for  non-academic  subjects  such  as 
commandant’s/commander's  time,  in-processing/out-processing, 
chaplain's  orientation,  etc.  Again,  these  types  of  modules 
can  be  identified  by  examining  the  modules  included  in  a 
comparable  course. 

After  the  modules  have  been  identified,  each  module  is 
assigned  a  name  which  summarizes  its  content. 

3.6.2  Identify  Methods  Within  Modules 

Table  3-21  displays  the  instructional  methods  which  are  used 
in  institutional  training.  The  instructional  methods  which 
mu^t  be  used  to  train  each  module  are  identified  by 
examining  the  instructional  methods  used  in  modules  from 
existing  courses  dealing  with  similar  tasks.  Methods  must 
be  added  or  deleted  to  reflect  the  differences  between  the 
tasks  included  in  the  module  from  the  comparable  course. 

3.6.3  Identify  Module  Training  Times 

Module  training  times  are  identified  in  a  two-step  process. 
In  the  first  step,  the  total  training  time  for  a  module  is 
determined.  This  is  accomplished  by  modifying  the  training 
time  from  the  comparable  existing  module  to  reflect  the 
differences  in  task  requirements  between  the  proposed  and 
existing  module. 
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TABLE  3-21  INSTRUCTIONAL  METHODS 


Audio  Tape 

Conference/Lecture 

Computer  Assisted  Instruction 

Case  Study 

Demonstration 

Dual  Flight  Hours  (only  aviator  courses) 

(do  not  include  in  ICH  computations) 

Hardware  Performance  Examination 
Nonhardware  Performance  Examination 
Nonhardware  Performance  Examination 
Elective  (in-house  only,  except  for  CGSC) 

Film 

Guest  Speaker 
Independent  Study 

Non-contact  Instruction  with  an  Instructor 
Available  in  Classroom 
Non-contact  Instruction  without  an 
Instructor  Available 

Hardware  Oriented  (hands-on)  Practical 
Application 

Nonhardware  Oriented  (non-classroom) 

Practical  Application 

Classroom  Practical  Application 

Programmed  Instruction  (using  programmed  text) 

Printed  Materials 

Besseler  Cue  See 

Seminar 

Solo  Flight  Hours  (only  aviator  courses) 

(do  not  include  in  ICH  computations) 

Simulation  Instruction 
Self-Paced  Instruction 
Slide  Tape 
Television 

Instructor  Led  Work  Group 
Student  Led  Work  Group 


Sources:  DA  Pam  570-558  Staffing  Guide 

for  U.S.  Army  Service  Schools 
and  TRADOC  Cir  351-12  Format 
for  Programs  of  Instruction 


In  the  second  step,  training  times  for  each  method  within  a 
module  ar  ^  determined.  This  is  accomplished  by  modifying 
the  training  times  for  the  methods  from  the  comparable 
existing  module  to  reflect  task  differences  between  the 
proposed  and  existing  module. 

3.6.4  Determine  Course  Length 

Total  course  length  is  determined  by  summing  the  module 
training  times. 

3.6.5  Document  Data 

Information  developed  in  the  previous  step  is  placed  in  the 
QPOI  format  listed  in  the  QPOI  Worksheet.  The  QPOI  format 
is  divided  into  two  general  parts.  The  first  part  provides 
summary  information  on  modules  within  each  course.  The 
second  part  provides  more  detailed  information  on  each 
module . 

After  the  QPOI  worksheets  ha”e  been  completed,  relevant  data 
from  these  worksheets  should  be  entered  into  the  SDT .  Table 
3-22  displays  the  most  desirable  index  for  entering  data 
into  the  SDT.  Additional  guidance  for  entering  data  into 
the  SDT  is  provided  in  the  SDT  User's  Guide. 

3.7  ASSIGN  TASKS  TO  MEDIA 

Procedures  for  assigning  tasks  to  media  are  described  in  the 
User's  Guide:  Media  Selection  Program.  This  program  is  an 
automated  aid  for  (1)  assigning  tasks  to  media,  and  (2) 
determining  the  efficiency  and  effectiveness  of  media-task 
combinations.  Relevant  data  from  the  Media  Selection 
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program  must  be  entered  into  the  SDT.  Table  3-23  displays 
the  most  desirable  order  for  entering  data  into  the  SDT. 
Additional  guidance  for  entering  data  into  the  SDT  is 
provided  in  the  SDT  User's  Guide. 

3.8  CONSTRUCT  TRAINING  PATHS 

OVERVI EW 

During  this  procedure,  training  paths  are  constructed  for 
each  of  the  MOSs  associated  with  the  BCS  and  New  system.  The 
training  paths  provide  a  convenient  framework  for  describing 
the  training  courses  required  to  train  each  MOS.  The 
training  paths  represent  the  possible  flow  of  individuals  as 
they  progress  along  the  career  path  in  their  MOS. 

PROCEDURE 

An  overview  of  the  procedure  for  constructing  training  paths 
is  provided  in  Figure  3-15.  The  first  step  in  this  procedure 
is  to  examine  the  non-system  related  courses  in  each  MOS 
which  were  identified  as  part  of  the  target  population 
identification  function  (Procedure  3.4).  The  courses  within 
each  MOS  are  then  sorted  by  skill  level.  When  this  is  com¬ 
plete,  the  courses  are  sequenced  to  reflect  task  and  skill 
learning  dependencies  (what  must  be  learned  before  what) 
within  skill  level.  Training  paths  are  then  developed  to 
represent  the  course  sequences  required  for  each  MOS.  An 
example  of  training  path  is  provided  in  Figure  3-16. 

A  worksheet  which  can  be  used  to  support  the  development  of 
training  paths  is  provided  in  Table  3-24.  Table  3-25 
displays  the  training  path  data  elements  which  must  be 
entered  into  the  SDT. 
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Figure  3-16.  Example  of  Training  Path  for  Institutional  Courses. 


XX-211B 

(WO) 


ATTENDANCE  IN  THE  13T30  COURSE  IS 
PREREQUISITE  FOR  THIS  COURSE. 


SUPERVISED 
ON-THE-JOB 
TRAINING 
v  (SOJT)  . 


THE  TRAINING  PROGRAM  BEYOND  EB 
WAS  NOT  PART  OF  THE  STUDY 


XXX-13T30 

(BTC) 


SUPERVISED 
ON-THE-JOB 
TRAINING 
v  (SOJT)  ^ 


PRIMARY 

LEADERSHIP 

COURSE 

(PLC) 


SUPERVISED 
ON-THE-JOB 
TRAINING 
v  (SOJT)  > 


SUPERVISED 
ON-THE-JOB 
TRAINING 
v  (SOJT) 


SUPERVISED 
ON-THE-JOB 
TRAINING 
v  (SOJT)  > 


SECTION  4.0 


ESTIMATE  TRAINING  RESOURCES 


OVERVIEW 


During  this  procedure,  training  resource  requirements  are 
determined  for  the  Predecessor,  Baseline  Comparison  (BCS), 
and  New  systems. 

PROCEDURE 


This  procedure  is  composed  of  six  lower  level  procedures 
(See  Figure  4-1).  During  the  first  procedure,  an  operating 
and  support  plan  is  developed  for  each  modified  or 
additional  course.  In  the  second  procedure,  the  number  of 
students  who  must  be  trained  in  each  course  is  determined. 
In  the  third  procedure,  the  number  of  instructors  required 
in  each  course  is  determined.  In  the  fourth  procedure, 
training  facilities  requirements  are  determined.  In  the 
fifth  procedure,  training  device  and  training  equipment 
requirements  are  determined.  In  the  sixth  procedure, 

requirements  for  other  types  of  training  resources  are 
determined.  Both  phased  and  steady-state  resource 

requirements  are  determined  during  this  procedure. 


Phased  resource  requirements  are  the  specific  resources  that 
are  required  while  a  weapon  system  is  being  phased  into  or 
out  of  the  inventory  (the  latter  case  is  generally  not 
relevant  during  systems  development).  Phased  resource 
requirements  are  displayed  by  calendar  year.  In  order  to 
determine  these  phased  resource  requirements,  information  on 
the  number  of  weapons  systems  introduced  to  the  field  in 
each  calendar  year  must  be  obtained  or  estimated. 
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Figure  4-1.  Overview  Diagram:  Determine  Course  Operating  and  Support  Plan 


Steady-state  resource  requirements  are  the  estimated  average 
yearly  resource  requirements  that  are  needed  to  maintain  the 
weapon  system  in  the  field  once  it  has  been  completely 
fielded.  To  estimate  steady-state  resource  requirements, 
all  that  is  required  is  an  estimate  of  the  total  number  of 
systems  that  will  be  in  the  field  once  system  installation 
has  been  completed.  Steady-state  requirements  are  also 
presented  in  yearly  units.  Because  they  require  less 
information  to  calculate,  only  steady-state  requirements  are 
generally  calculated  during  the  very  earliest  phases  of  the 
acquisition  process. 

4.1  DEVELOP  COURSE  OPERATING  AND  SUPPORT  PLAN 

OVERVI EW 

During  this  procedure,  an  operating  and  support  plan  is 
developed  for  each  new  or  modified  course  in  the  BCS  and  New 
systems.  The  operating  and  support  plan  specifies  (a) 
course  locations,  (b)  course  frequency,  and  (c)  usage  rates 
for  training  recruits. 

PROCEDURE 

This  procedure  consists  of  the  following  three  steps:  (1) 
determine  course  location,  (2)  determine  course  frequency, 
and  (3)  determine  usage  rates.  Worksheets  which  can  be  used 
to  support  this  procedure  are  provided  in  Figures  4-2  and 
4-3. 

4.1.1  Determine  Course  Location 

An  overview  of  the  process  for  identifying  course  locations 
is  provided  in  Figure  4-4.  MOS  and  setting  are  the  primary 
factors  for  determining  course  location.  Since  these 


Steady  State 


Figure  4-2.  Operating  and  Support  Plan  Worksheet  (OPS). 


/  ✓ 


information  items  are  iden t i f i ed  as  part  of  the  QPOI 
construction  process  (Procedure  3.6),  identification  of 
course  locations  is  relatively  straightforward.  Additional 
guidance  for  course  location  identification  can  be  provided 
by  examining  (a)  the  course  locations  used  in  comparable 
existing  courses  and  (b)  the  projected  locations  where  the 
weapon  system  will  be  fielded. 

4.1.2  Determine  Course  Frequency 


Course  frequency  (i.e.,  the  number  of  starts  and  sessions 
per  site)  is  determined  by  applying  the  algorithm  listed  in 
Table  4-1.  To  estimate  course  frequency,  you  must  determine 
the  number  of  students  to  be  trained  (Procedure  4.2).  The 
algorithm  listed  in  Table  4-1  must  be  applied  for  each 
location  and  year  in  which  the  course  will  be  given. 


4.1.3  Determine  Usage  Rates 

Usage  rates  are  determined  for  instructors  and  training 
media.  The  usage  rate  for  instructors  is  called  the 
student/instructor  ratio.  The  usage  rate  for  media  is 
called  the  student/media  ratio. 

Student/instructor  ratios  are  determined  for  each 
instructional  method  used  in  a  course.  Table  4-2  displays 
the  student/instructor  ratios  that  are  provided  for  each 
type  of  instructional  method  in  DA  Pam  570-558  and  TRADOC 
Cir  351-12. 


Student/media  rates  are  determined  by  identifying  the  rates 
that  were  used  for  similar  types  of  media  in  comparable 
courses.  This  information  may  be  obtained  by  examining  the 


TABLE  4-1.  ALGORITHM  FOR  DETERMINING  NUMBER  OF  COURSE 
STARTS  AND  SESSIONS  PER  SITE. 


1.  Establish  number  of  courses  required  per  site 


f  NMAX  =  NCOURSE 


=  Student  Input  Requirements 
=  Number  of  Training  Sites 
=  Maximum  Student-Instructor  Ratio 
for  Course 

=  Number  of  Courses  Required  Per 
Site 

2.  Determine  maximum  number  of  starts  per  year 
(CMAX) 

- — -  =  CMAX 

CLENGTH 

CLENGTH  =  Course  Length  in  weeh 

CMAX  =  Maximum  Number  of  Course  Starts 

per  year  (NCON) 


3.  Select  the  smaller  of  NCOURSE  or  CMAX  as  the 
number  of  course  starts  (NCON)  required  per 
site . 


.  STUDIN 
1  NSITE 


STUDIN 

NSITE 

NMAX 

NCOURSE 


4.  Determine  maximum  number  of  sessions  needed  per 
start  (NSES) 


If 

NCOURSE 

CMAX, 

NSES  - 

If 

NCOURSE 

CMAX, 

NCOURSE 

MAX 

Round  NSES  up  to  next  highest  unit. 
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TABLE  4-2.  METHODS  OF  INSTRUCTION  AND  ASSOCIATED 
STUDENT/INSTRUCTOR  RATIOS 


AT 

C 

CAI 

CS 

D 

DF 

El 

E2 

E3 

EL 

F 

GS 

IS 

NCI 

NC2 

PEI 

PE2 

PE3 

PI 

PM 

QC 

S 

SF 

SI 

SP 

ST 

TV 

WC1 

WC2 


Audio  Tape 

Conference /Lecture 

Computer  Assisted  Instruction 

Case  Study 

Demonstration 

Dual  Flight  Hours  (Only  Aviator  Courses) 
Hardware  Performance  Examination 
Nonhardware  Performance  Examination 
Nonhardware  Performance  Examination 
Elective  (In-House  Only,  Except  for 
CGSC ) 

Film 

Guest  Speaker 
Independent  Study 
Non-contact  Instruction  with  an 
Instructor  Available  in  Classroom 
Non-contract  Instruction  without  an 
Instructor  Available 
Hardware  Oriented  (Hands-On)  Practical 
Application 

Hardware  Oriented  (Non-Classroom) 
Practical  Application 
Classroom  Practical  Application 
Programmed  Instruction  (Using  Pro¬ 
grammed  Text) 

Printed  Materials 
Besseler  Cue  See 
Seminar 

Solo  Flight  Hours  (Only  Aviator  Courses) 

Simulation  Instruction 

Self-Paced  Instruction 

Slide  Tape 

Television 

Instructor  Led  Work  Group 
Student  Led  Work  Group 


20:1 

1  per  class 
20:1 
20:1 
20  :  1 

6:1 

6:1 

1  per  class 

1  per  class 
1  per  class 
1  per  class 
Local  Appraisal 


6:1 

6:1 

20:1 

20:1 
20:1 
20:  1 
20  :  1 

Local  Appraisal 
20:1 
20:1 

1  per  class 
Local  Appraisal 
Local  Appraisal 


Sources:  DA  PAM  570-558 

Staffing  Guide  for  U.S.  Army  Service  Schools 
TRADOC  Cir  351-12 

Format  for  Programs  of  Instruction 
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POIs  and  other  associated  course  documentation  from  the 


comparable  course.  Student/media  rates  should  not  exceed 
the  student/instructor  ratio  for  the  instructional  methods 
with  which  they  are  associated. 

4.1.4  Document  Data 

As  course  frequency  and  location  information  is  developed, 
it  should  be  documented  in  the  operating  and  support  plan 
worksheets  (see  Figure  4-2).  Media  usage  rates  should  be 
documented  in  the  Media  Usage  Rate  Worksheet  (see  Figure  4- 
3).  Student/instructor  usage  rates  should  be  documented  in 
the  QPOI  Part  I  worksheet  which  is  listed  in  Procedure  3.6. 

After  the  worksheets  have  been  completed,  relevant  data  from 
these  worksheets  should  be  entered  into  the  SDT.  Table  4-3 
describes  the  most  desirable  order  for  entering  data  into 
the  SDT.  Additonal  guidance  for  entering  data  into  the  SDT 
is  provided  in  the  SDT  User’s  Guide. 

4.2  DETERMINE  NUMBER  OF  STUDENTS  TO  BE  TRAINED 

OVERVIEW 

During  this  procedure,  the  number  of  students  to  be  trained 
in  the  steady-state  and  phased  situations  is  determined. 

PROCEDURE 

An  algorithm  for  determining  the  number  of  students  to  be 
trained  in  the  steady-state  condition  is  provided  in  Table 
4-4.  A  worksheet  which  can  be  used  in  support  of  this 
algorithm  is  provided  in  Table  4-5.  A  worksheet  which  can 
be  used  to  calculate  the  number  of  students  to  be  trained  in 
the  phased  condition  is  presented  in  Table  4-6.  A  worksheet 
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Table  4-4.  Algorithm  for  Determining  Steady-State 
Training  Input  Requirements 


Inflate  the  given  manpower  requirements  by  the  TTHS  rates: 

M  =  MR  x  (1  +  T  ) 
n  n  n 

where  Mn  =  Adjusted  manpower  requirements  for  paygrade  n 
MRn  =  Manpower  requirements  for  paygrade  n 
Tn  =  Trainee,  Transient,  Holdee,  and  Student  rates. 

Calculate  levels  of  X  which  satisfy  the  following  equations 
for  all  n:  n 


X  x  (A  +  U  )  —  X  .  x  U  i  ,  and  X„  =  M  +  C> 
n  n  n  n— 1  n— l  n  n  n 

where  Xfi  =  Personnel  requirements  for  paygrade  n 

An  =  Yearly  attrition  rate  for  paygrade  n 

Un  =  Yearly  upgrade  rate  for  paygrade  n 

0n  =  Overhead  rate  for  paygrade  n. 

Calculate  the  number  of  personnel  to  be  trained  per  year: 

TR  =  Xn  x  (A  +  U  ) 
n  n  n  n 

where  TR  =  Yearly  replacement  personnel  to  be  trained 
for  paygrade  n. 


Table  4-5.  Number  of  Students-Steady  State  Worksheet  (NSSS) 


TOTAL 


BLOCK  THREE 


TABLE  4-6.  WORKSHEET  FOR  PHASED  COURSE  INPUT  REQUIREMENTS  (cont'd) 


INSTRUCTIONS  FOR  COMPUTING  ANNUAL  TRAINING  INPUT 

REQUIREMENTS 

This  guide  provides  the  procedural  steps  to  be  used  in  com¬ 
puting  the  final  adjusted  annual  training  input  requirements  to 
a  single  course  of  instruction.  The  development  of  training  in¬ 
put  requirements  involves  the  application  of  adjustments  to  the 
number  of  personnel  required  to  be  trained  in  order  to  compensate 
for  losses  through  rating  and  school  attrition  and  to  accomodate 
Transient,  Trainee,  Holdee,  and  Student  (TTHS)  requirements. 

PROCEDURAL  STEPS 

STEP  1.  BLOCK  ONE,  COLUMN  (1) 

Enter  the  fiscal  years  of  the  planning  period. 

STEP  2 .  BLOCK  ONE ,  COLUMN  ( 2 ) 

Enter  the  cumulative  number  of  trained  personnel  required. 

STEP  3.  BLOCK  ONE,  COLUMN  (3) 

Enter  the  product  of  the  Total  Unadjusted  Trained  Personnel 
required  in  Column  (2) and  the  TTHS  factor  to  determine  the 
number  of  TTHS  Adjusted  Trained  Personnel  Required. 

Transient,  Trainee ,  Holdee,  and  Student  ( TTHS )  Factor  -  a 
factor  calculated  from  historical  data  used  to  inflate  the 
cumulative  total  trained  personnel  to  account  for  personnel 
in  a  transient,  trainee,  holdee,  or  student  status  who  are 
therefore  unavailable  to  occupy  a  duty  position.  A  factor 
of  1.09  should  be  used  in  the  absence  of  a  historical  figure. 
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TABLE  4-6.  Worksheet  for  Phased  Course  Input  Requirements  (cont'd) 


STEP  4.  BLOCK  TWO,  COLUMN  (1) 

Enter  the  fiscal  years  affected  from  Block  One,  Column  (1). 

STEP  5.  BLOCK  TWO,  COLUMN  (2) 

Enter  the  TTHS  Adjusted  Trained  Personnel  Required  from 
Block  One,  Column  (3)  as  the  Starting  Inventory  for  the 
following  fiscal  year .  These  data  are  entered  one  year 
later  since  the  number  trained  in  one  year  represents  the 
starting  inventory  for  the  next  year. 

STEP  6 .  BLOCK  TWO ,  COLUMN  ( 3 ) 

Enter  the  product  of  the  starting  inventory  in  Column  (2) 
and  the  Loss  Factor  to  determine  the  number  of  losses  per 
year. 

Loss  Factor  -  a  factor  calculated  from  historical  data  used 
to  inflate  the  number  of  personnel  required  to  be  trained 
to  account  for  losses  due  to  non-reenlistment  and  other 
attrition  from  the  service.  Loss  is  a  function  of  several 
different  loss  factors;  a  factor  of  0.15  should  be  used 
in  the  absence  of  an  empirical  figure. 

STEP  7.  BLOCK  THREE,  COLUMN  (1) 

Enter  the  fiscal  years  affected  from  Block  One,  Column  (1). 

STEP  8.  BLOCK  THREE,  COLUMN  (2) 

Enter  the  TTHS  Adjusted  Trained  Personnel  Required  from 
Block  One,  Column  (3). 

STEP  9.  BLOCK  THREE,  COLUMN  (3) 

Enter  the  Starting  Inventory  from  Block  Two,  Column  (2). 

STEP  10.  BLOCK  THREE,  COLUMN  (4) 

Enter  the  difference  between  TTHS  Adjusted  Trained  Personnel 
Required  in  Column  (2)  and  Starting  Inventory  in  Column  (3) 
to  determine  Training  Growth.  The  Training  Growth  may  be 
positive  or  negative. 

Training  Growth  -  the  difference  between  training  requirements 
and  starting  inventory  between  one  fiscal  year  and  another. 
Inventory  in  excess  of  requirements  will  result  in  a  negative 
growth;  however,  growth  is  normally  positive. 

STEP  11.  BLOCK  THREE,  COLUMN  (5) 

Enter  losses  in  the  appropriate  fiscal  year  from  Block  Two, 
Column  { 3 ) . 
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TABLE  4-6.  Worksheet  for  Phased  Course  Input  Requirements  (ccnt'd) 


STEP  12.  BLOCK  THREE,  COLUMN  (6) 

Enter  the  sum  of  Training  Growth  from  Column  (4)  and  Losses 
from  Column  (5)  to  determine  Course  Output  Requirements. 

STEP  13.  BLOCK  THREE,  COLUMN  (7) 

Enter  the  product  of  Course  Output  Requirements  from 
Column  (6)  and  the  Course  Attrition  Factor  to  determine 
Course  Attrition. 

Course  Attrition  Factor  -  a  factor  calculated  from  historical 
data  used  to  inflate  the  Course  Output  Requirements  to 
account  for  trainees  who  fail  to  complete  a  course  of  in¬ 
struction  for  a  variety  of  reasons.  Course  Attrition  Factors 
differ  from  course  to  course;  however  a  factor  of  0.10  should 
be  used  in  the  absence  of  an  empirical  figure. 

STEP  14.  BLOCK  THREE,  COLUMN  (8) 

Enter  the  sum  of  Course  Output  Requirements  from  Column  (6) 
and  Course  Attrition  from  Column  (7)  to  determine  Adjusted 
Course  Input  Requirements. 

STEP  15.  BLOCK  THREE,  COLUMN  (95 

Enter  the  Backout  in  appropriate  (+)  or  (-)  column  in  ac¬ 
cordance  with  the  following  procedure. 

a.  Compute  the  Backout  Factor  as  a  percentage 
by  multiplying  course  length  in  weeks  by 
two  (2).  (Note:  This  number  is  not  entered 
on  the  worksheet.) 

b.  Multiply  the  Backout  Factor  by  the  Adjusted 
Course  Input  Requirements  for  the  second 
fiscal  year  in  Column  (8).  A  Backout  Factor 
is  not  computed  for  the  first  fiscal  year. 

c.  Enter  the  backout  computed  in  b.  as  a  posi¬ 
tive  number  for  the  first  planning  year  in 
Column  (9)  and  as  a  negative  number  for  the 
second  planning  year  in  Column  (9).  This 
figure  is  a  double  entry  as  it  represents 
both  an  additional  input  for  the  preceeding 
fiscal  year  and  a  backout  for  the  current 
fiscal  year. 

d.  Repeat  steps  b.  and  c.  above  for  each  fiscal 
year  requiring  Backout. 

Backout  Factor  -  a  calculated  figure  applied  to  the  Adjusted 
Course  Input  Requirements  to  account  for  overlapping  of  a 
course  from  one  fiscal  year  to  another.  Courses  with  lengths 
in  excess  of  two  weeks  and  convened  on  a  continuous  basis  re- 


TABLE  4-6.  Worksheet  for  Phased  Course  Input  Requirements  (cont'd) 


quire  a  portion  of  the  input  to  be  admitted  the  previous 
fiscal  year  in  order  to  meet  the  required  output  of  the 
following  year.  The  Backout  Factor  is  a  percentage  equal 
to  twice  the  course  length  and  is  applied  to  each  fiscal 
year  with  the  exception  of  the  first.  The  Backout  Factor 
is  not  applied  to  courses  less  than  two  weeks  in  length. 

STEP  16.  BLOCK  THREE,  COLUMN  (10) 

Enter  the  sum  of  the  Adjusted  Course  Input  Requirements 
from  Column  (8)  and  the  net  Backout  in  Column  (9)  to  de¬ 
termine  the  Final  Adjusted  Annual  Training  Input  Require¬ 
ments  . 


The  example  on  the  following  page  uses  a  Starting  Inventory  of 
zero,  a  four  week  course  length,  and  the  following  factors  and  un¬ 
adjusted  requirements  by  fiscal  year: 


TOTAL  UNADJ  TRAINED  PERS  REQ ' D 

61 
103 
148 
183 
196 
196 


TTHS  factor:  1.09 
Loss  factor:  .15 

Course  Attrition  Factor:  .10 


FY88 

FY89 

FY90 

FY91 

FY92 

FY93 
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Table  4-6.  Worksheet  for  Phased  Course  Input  Requirements  (Continued) 


BLOCK  THREE 


which  can  be  used  to  document  the  results  of  the  steady- 
state  and  phased  algorithm  applications  is  provided  in  Table 

4-7. 

After  the  worksheets  have  been  completed,  relevant  data  from 
these  worksheets  must  be  entered  into  the  SDT.  Table  4-8 
displays  the  most  desirable  sequence  for  entering  data  into 
the  SDT. 

4.3  DETERMINE  THE  NUMBER  OF  INSTRUCTORS 

OVERVIEW 

During  this  procedure,  the  number  of  instructors  required  in 
each  course  is  determined. 

PROCEDURE 

An  algorithm  for  determining  the  number  of  instructors  is 
presented  in  Table  4-9.  This  algorithm  has  been  automated 
in  the  ETES  Resource  and  Cost  Estimation  Techniques  (see 
User's  Guide:  Resource  and  Cost  Estimation  Techniques).  The 
algorithm  should  be  applied  for  each  location  and  year  in 
which  the  course  is  given.  A  worksheet  for  documenting  the 
results  of  the  RCET  is  contained  in  Table  4-10.  Table  4-11 
displays  guidelines  for  entering  data  into  the  SDT. 

4.4  DETERMINE  FACILITIES  REQUIREMENTS 

OVERVIEW 

During  this  procedure,  facilities  requirements  are 
determined. 
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Table  4-7.  Number  of  Students  To  Be  Trained  Worksheet  (NSTUD) 


TOTAL 


TABLE  4-9.  ALGORITHM  FOR  DETERMINING 
NUMBER  OF  INSTRUCTORS 


Determine  the  number  of  Instructor  Contact  Hours 
(ICH)  per  class. 

NM 

ICH  per  _  £  hours  of  method  i  x  class  size 
class  <i  student/instructor  ratio  for  method 
i=l 

where  NM  is  the  number  of  methods. 


Determine  the  number  of  instructors. 

number  of  _  ICH  per  „  course  frequency 

instructors  class  1250 


TOTAL 


PROCEDURE 


An  algorithm  for  determining  facilities  requirements  is 
displayed  in  Table  4-12.  A  worksheet  for  applying  the 
algorithms  to  a  specific  course  location  and  year  is 
contained  in  Table  4-13.  A  worksheet  for  summarizing  the 
results  for  a  specific  course  is  displayed  in  Table  4-14. 

4.5  DETERMINE  TRAINING  DEVICE  AND  TRAINING  EQUIPMENT 
REQUIREMENTS 

OVERVIEW 

During  this  procedure,  resource  requirements  are  determined 
for  training  devices  and  training  equipments. 

PROCEDURE 

An  algorithm  for  determining  the  number  of  training  devices 
is  presented  in  Table  4-15.  A  worksheet  for  applying  these 
algorithms  is  displayed  in  Table  4-16. 

It  should  be  noted  that  a  training  device  can,  by  itself,  be 
considered  a  large  system  acquisition  with  its  own  asso¬ 
ciated  resource  requirements.  Estimation  of  these  addi¬ 
tional  resource  requirements  is  beyond  the  current  scope  of 
ETES.  However,  it  is  expected  that  these  resource  require¬ 
ments  could  be  estimated  using  comparability  analysis. 
Input  to  the  determination  of  training  device  resource 
requirements  is  provided  by  the  task  requirements  associated 
with  each  device  which  are  determined  in  Procedure  2.0  and 
the  instructional  programs  developed  during  Procedure  4.5. 


Table  4-12.  Algorithm  For  Determining 

Training  Facilities  Requirements 

FR.  =  MST  x  FRS . 

i  i 

where  FR^  is  the  total  square  footage  of  facilities  requirements 
required  for  the  it^'  facility  type,  MST  is  the  maximum  student 
throughput  and  FRS_^  is  the  number  of  square  feet  required  per  student 
for  the  i^  type  of  facility. 

Maximum  student  throughput  (MST)  is  calculated  by  the  following 
equation 

MST  =  CS  x  MNC 

where  CS  is  the  class  size  and  MNC  is  the  maximum  number  of  courses 
conducted  concurrently  during  the  year. 


If  FRS^  is  not  available,  it  may  be  estimated  through  comparability 
analysis.  More  specifically,  it  can  be  calculated  as  follows: 


FRS± 


TSF . 

_ l 

CS  x  MNC 


where  TSF^  is  the  total  square  footage  in  the  comparable  course  for 
the  i*"*1  type  of  facility,  CS  is  the  class  size  for  the  comparable 
course,  and  MNC  is  the  maximum  number  of  classes  conducted  concurrently 
for  the  comparable  course. 
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Table  4-14.  Facilities  Requirements  Worksheet  (NFAC) 


TOTAL 


Table  4-15.  Algorithm  for  Determining  Number 

of  Training  Devices/Equipments  Per  Site. 


Determine  number  of  hours  to  be  spent  on  device 
per  day. 


NSES  * 


,  NMAX 
NSTUD 


NCHOURS 
(  CLENGTH 


NHRSD 


NSES 

NMAX 

NCHOURS 

CLENGTH 

NSTUD 

NHRSD 


Maximum  number  of  sessions  per  start 
Maximum  student/instructor  ratio  for  course 
Number  of  hours  to  be  spent  on  device  for 
each  student 
Course  length ,  in  days 

Number  of  students  who  can  be  trained  on 
device  simultaneously 

Number  of  hours  to  be  spent  on  device  per  day 


2. 


Determine  number  of  devices  required. 


NHRSD 

SHIFT 


NDEVICE 


NHRSD  =  Number  of  hours  spent  on  device  per  day 
SHIFT  =  Maximum  operating  hours  per  day  per  device 
NDEVICE  =  Number  of  devices 


3. 


Round  NDEVICE  TO  NEXT  HIGHEST  INTEGER 


Worksheet  For  Determining  Number  of  Training  Devices/Equipments  (TDEV) 


The  same  algorithm  used  for  training  devices  can  be  used  for 
training  equipment.  Use  of  actual  weapon  system  equipment 
for  training  will  generate  additional  resource  requirements. 
Again,  determination  of  these  resource  requirements  is 
beyond  the  present  scope  of  ETES.  However  it  is  expected 
that  comparability  analysis  could  be  used  to  determine  such 
additional  resource  requirements. 

A  worksheet  for  summarizing  the  results  of  the  application 
of  the  training  device/equipment  algorithms  is  provided  in 
Table  4-17.  Table  4-18  displays  guidelines  for  entering 
data  into  the  SDT. 

4.6  DETERMINE  REQUIREMENTS  FOR  OTHER  TRAINING  RESOURCES 

OVERVIEW 

During  this  procedure,  resource  requirements  for  media  and 
instructional  aids  and  ammunition  are  determined. 

PROCEDURE 

Procedures  for  determining  resource  requirements  for  media 
instructional  aids  and  ammunition  are  provided  in  the 
sections  which  follow. 

4.6.1  Determine  Number  of  Training  Aids/Instruct ional  Media 

There  are  two  general  algorithms  for  determining  the  number 
of  instructional  aids  and  other  forms  of  instructional  media 
directly  assignable  to  a  course:  one  algorithm  for  deter¬ 

mining  the  number  studen t- i ssued  media  and  another  algorithm 
for  determining  the  number  non-student- i ssued  media. 
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TOTAL 


Student-Issued  Media 

Media  issued  to  students  (i.e.  ,  students  consume  the  media 
during  the  course  or  take  it  with  them  when  they  leave)  is 
determined  by  the  following  equation: 

TMRi  -  NS  x  URi 

where  TMR^  is  the  total  media  requirements  por  the  ith 
medium,  NS  is  the  number  of  students  taking  the  ^ourse  per 
year,  and  UR^  is  the  number  of  the  ith  media  issued  to  each 
student  (this  number  is  usually  one). 

Non-Student  Issued  Media 

Media  which  is  not  issued  to  students  may  be  determined  by 
the  following  equation: 

TMRt  =  MST  X  URi 

where  TMR^  is  the  total  resource  requirements  for  the  ith 
course,  MST  is  the  maximum  student  throughout  the  course  and 
UR  ^  is  the  usage  rate  for  the  ith  media.  Maximum  student 
throughput  (MST)  is  calculated  by  the  following  equation: 

MST  =  CS  x  MNC 

where  CS  is  the  class  size  and  MNC  is  the  maximum  number  of 
courses  conducted  concurrently  during  the  year.  A  worksheet 
for  determining  the  number  of  media  and  instructional  aids 
is  presented  in  Table  4-19.  A  worksheet  for  summarizing  the 
results  of  this  algorithm  is  presented  in  Table  4-20.  Table 
4-21  displays  guidelines  for  entering  data  into  the  SDT. 
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Table  4-19.  Worksheet  for  Determining  Number  of  Media  and  Instructional  Aids  (NMED). 


Table  4-20.  Number  of  Media/Aids  (MEDSUM). 


TOTAL 


4.6.2  Determine  Ammunition  Requirements 


To  determine  ammunition  requirements  for  developing  systems 
during  the  early  phases  of  the  acquisition  process,  compar¬ 
ability  analysis  procedures  must  be  employed.  More  specifi¬ 
cally,  comparable  courses  must  be  identified  and  data  on  the 
ammunition  requirements  for  these  courses  must  be  obtained. 
This  data  should  be  available  in  the  ammunition  annex  in  the 
Program  of  Instruction  (POI)  for  each  course.  Ammunition 
requirements  for  a  comparable  course  should  be  modified  to 
reflect  the  task,  methods/media,  and  operating  and  support 
plan  differences  between  a  comparable  course  and  a  new 
course.  A  comparable  course  should  have  been  identified 
during  the  POI  instruction  process  (see  Procedure  3.6). 


Table  4-21  lists  a  worksheet  which  can  be  used 
the  modification  of  ammunition  data  from  a 
course . 


to  support 
comparable 


Table  4-22  lists  a  worksheet  which  can  be  used  to  summarize 
ammunition  requirements.  Table  4-23  provides  guidelines  for 
entering  data  into  the  SDT . 
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Table  4-21.  Ammunition  Requirements  Worksheet  (AMREQ) 


Table  4-22.  Ammunition  Requirements  Summary  Sheet  (AMSUM). 
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SECTION  5 


DETERMINE  TRAINING  COST 


OVERVIEW 

During  this  procedure,  costs  are  estimated  for  the  BCS  and 
New  system  courses,  and  aggregate  cost  estimates  are 
developed  for  the  Predecessor,  BCS,  and  New  systems. 

PROCEDURES 

An  overview  of  the  procedures  in  this  step  is  provided  in 
Figure  5-1. 

5.1  ESTIMATE  INDIVIDUAL  COURSE  COST 

OVERVIEW 

During  this  procedure,  costs  are  estimated  for  the  BCS  or 
New  system  courses.  A  worksheet  which  can  be  used  to 
support  this  procedure  is  provided  in  Figure  5-2. 

PROCEDURE 

The  first  step  in  this  procedure  is  to  obtain  the  cost  per 
graduate  for  each  course  and  to  enter  this  information  into 
column  (a)  of  the  ICOST  worksheet.  For  BCS  or  New  system 
courses  which  are  unmodified  existing  courses,  the  cost  per 
graduate  can  be  obtained  directly  from  the  Cost  Analysis 
Program  -  MOS  Training  Costs  ( RCSATRM-1 59R1 ) .  Outputs  from 
this  program  may  be  obtained  from  the  Resource  and  Economic 
Analysis  OPC,  ATRM-R,  in  the  DCS  Resource  Management  -  ATRM 
at  TRADOC  Headquarters.  All  course  costs  should  be  obtained 
from  the  latest  version  of  the  cost  analysis  program. 
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Figure  5-1.  Overview  Diag 
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Determine  Training  Cost  (5.1) 


Individual  Course  Cost  Worksheet  (ICOST). 


Cost  per  graduate  must  be  estimated  for  any  modified  or  new 
course  which  is  part  of  either  the  BCS  or  New  system.  These 
estimates  can  be  obtained  by  applying  the  automated  ETES 
Resource  and  Cost  Estimation  Technique  (RCET).  Using  RCET, 
costs  for  a  new  or  modified  course  are  estimated  by 
identifying  a  comparable  existing  course,  obtaining  cost 
data  for  this  course  from  the  ATRM  system  and  then  modifying 
this  data  to  reflect  the  differences  in  key  resource 
requirements  (for  example,  number  of  instructors)  between 
the  comparable  and  new  course.  An  overview  of  RCET  is 
provided  in  Appendix  B.  A  more  detailed  description  of  RCET 
is  provided  in  the  User's  Guide:  RCET. 

Once  the  cost  per  grad  has  been  determined  for  each  course, 
the  yearly  number  of  students  to  be  trained  for  both  the 
steady  state  and  phased  conditions  must  be  taken  from  the 
NSTD  worksheet  which  was  completed  in  Procedure  4.2  and 
entered  into  the  appropriate  columns  in  the  ICOST  worksheet. 
Yearly  costs  can  then  be  calculated  by  multiplying  the  cost 
per  grad  (from  column  (a))  by  the  number  of  grads  in  each 
year. 

Total  costs  per  course  can  be  obtained  by  summing  over  the 
years  in  which  the  course  will  be  taught  (do  not  include 
steady  state  costs  in  this  calculation).  In  some 
situations,  it  may  not  be  possible  to  estimate  the  phased 
graduate  requirements  (see  Section  4.2).  In  these  cases,  it 
is  not  possible  to  estimate  phased  course  costs. 

It  should  be  noted  that  the  cost  estimates  developed  during 
this  step  are  only  designed  to  provide  input,  for  training 
evaluation;  these  costs  may  not  be  appropriate  for  budgeting 
est imates . 


After  the  ICOST  worksheets  have  been  completed,  relevant 
data  form  these  worksheets  should  be  entered  into  the  SDT. 
Table  5-1  describes  the  most  desirable  order  for  entering 
data  into  the  SDT.  Additional  guidance  for  entering  data 
into  the  SDT  is  provided  in  the  SDT  User's  Guide. 

5.2  AGGREGATE  COURSE  COSTS 

OVERVIEW 

During  this  step,  courses  costs  are  aggregated  across  MOSs 
to  provide  total  cost  measures  for  the  Predecessor,  BCS,  and 
New  systems.  A  worksheet  (ACOST)  which  can  be  used  to 
support  this  procedure  is  provided  in  Figure  5-3. 

PROCEDURE 

Total  steady  state  course  costs  for  each  MOS  are  obtained 
directly  from  the  subcost  total  under  column  (c)  of  the 
ICOST  worksheet  and  entered  into  the  appropriate  column  in 
the  ACOST  worksheet. 


Similarly,  the  phased  costs  per  year  are  obtained  from  the 
subtotals  listed  in  column  (m)  of  the  ICOST  worksheet  and 
entered  into  the  appropriate  column  in  the  ACOST  worksheet. 
These  costs  are  then  summed  across  MOSs  to  provide  a  total 
cost  value  for  the  Predecessor,  BCS  and  New  systems. 


Figure  5-3.  Course  Aggregation  Worksheet  (ACOST) 


SECTION  6.0  -  ESTIMATE  TRAINING  EFFICIENCY/EFFECT  I 


During  this  procedure,  training  "efficiency"  raeasi 
developed  for  three  training  program  elements:  (1)  m 
training  devices  (the  most  expensive  *-ype  of  medi 
training  methods  (see  Figure  6-1).  In  addition,  e 
of  the  effectiveness  of  an  entire  training  prog 
developed . 

6.1  ESTIMATE  TRAINING  EFFICIENCY 

OVERVIEW 

Training  efficiency  provides  a  measure  of  how  w 
characteristics  of  the  tasks  to  be  trained  mai 
characteristics  of  a  training  program  element  (for  < 
media), 1  Training  efficiency  measures  are  easily  esi 
even  during  the  early  phases  of  the  acquisition  proc< 

PROCEDURE 

An  overview  of  this  procedure  is  presented  in  Figu 
three  efficiency  measures  are  developed  durin 
procedure  (training  media  efficiency,  training 
efficiency,  and  training  method  efficiency).  Me 
method  efficiency  are  assessed  using  procedures  f 
Training  Efficiency  Estimation  Model  (TEEM). 


1  The  concept  of  training  efficiency  (congruen 
task  character  ist  ics )  was  initially  developed  by  Jc. 
(1979)  in  the  TEEM  project. 


Figure  6-1  Overview  Diagram:  Estimate  Training  Efficiency/Effectiveness 
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device  efficiency  is  measured  using  procedures  from  the 
TRAINVICE  methodology. 


6.1.1  Estimate  Media  Efficiency 

OVERVIEW 

To  estimate  training  efficiency  for  media,  procedures  from 
the  TEEM  model  are  employed.  These  procedures  have  been 
automated  in  the  ETES  Media  Selection  Program.  An  overview 
of  the  Media  Selection  Program  is  provided  in  Appendix  A.  A 
detailed  description  of  the  program  is  provided  in  the 
User's  Guide:  Media  Selection  Program.  This  guide  contains 
a  detailed  description  of  the  procedures  for  estimating 
media  efficiency. 

PROCEDURE 

An  overview  of  the  procedure  for  estimating  media  efficiency 
is  presented  in  Figure  6-3.  To  begin  the  procedure,  tasks 
are  rated  on  a  set  of  variables  describing  the  stimulus, 
response,  and  feedback  variables.  These  ratings  indicate 
which  variables  can  be  applied  to  each  task  (indicated  by 
assigning  a  1  or  0  to  each  task-variable  combination).  The 
maximum  possible  match  that  is  appropriate  for  each  task  is 
then  determined  by  adding  up  the  number  of  l's.  (The 
applicability  of  task  variables  to  each  media  category  has 
already  been  determined  and  has  been  incorporated  into  the 
Media  Selection  Program). 

The  match  between  media  and  tasks  is  determined  by  counting 
the  number  of  psychological  variables  they  have  in  common. 
Training  efficiency  of  each  media  is  determined  by  dividing 
the  actual  match  by  the  maximum  possible  match. 
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Figure  6-3.  Overview  Diagram:  Determine  Training  Efficiency 
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As  an  added  feature,  each  task  can  be  weighted  by  its  task 
criticality  during  the  calculation  of  efficiency  using  the 
following  formula: 


E  . 

1 


n  m 
I  L 
j=l  k=l 


c  . 

1 


m 


i  jk 


where  Ex  is  the  efficiency  of  the  ith  MEDIUM,  Cj  is  the 
criticality  of  the  jth  task,  M  ^  .  is  the  match  between  the 
ith  medium  and  the  jth  task  on  the  kth  variable,  N  is  the 
number  of  tasks  assigned  to  the  medium,  and  M  is  the  number 
of  psychological  variables.  Procedures  for  calculating  task 
criticality  are  described  in  Procedure  3.1. 

It  should  be  noted  that  the  Media  Selection  Program  can  also 
be  used  to  assign  tasks  to  media  as  well  as  to  evaluate  the 
efficiency  of  any  given  assignment.  An  overview  of 
procedures  for  using  the  program  to  make  media  assignments 
is  presented  in  Procedure  3.7. 

6.1.2  Estimate  Training  Device  Efficiency 

OVE  RV I EW 


Training  devices  are  often  the  most  expensive  type  of 
training  media  to  provide  more  refined  estimates  of  the 
efficiency  of  this  critical  type  of  training  device, 
procedures  from  the  TRAINVICE  methodology  may  be  employed. ^ 


A  detailed  description  of  TRAINVICE  is  provided  in  the 
Training  Device  Requirements  Guide. 


PROCEDURE 


Process:  An  overview  of  the  steps  in  the  TRAINVICE 
procedure  is  presented  in  Figure  6-4.  First,  a  measure  of 
task  commonality  (TC)  is  constructed  by  counting  the  number 
of  tasks  which  can  be  trained  on  the  device.  Second,  a 
measure  of  physical  similarity  (PS)  is  constructed  by  rating 
how  well  the  physical  features  of  the  controls  and  displays 
of  the  device  match  the  features  on  the  control  and  devices 
of  the  operational  equipment.  Third,  a  measure  of  functional 
similarity  ( FS )  is  constructed  by  rating  how  well  the  infor¬ 
mation  flow  presented  to  students  in  each  control  displays 
matches  the  information  flow  for  the  same  controls  and 
displays  in  the  operational  equipment.  Fourth,  the  improve¬ 
ment  in  skill  and  knowledge  (SK)  which  can  be  expected  for 
each  device  is  determined.  Fifth,  the  task  training 
difficulty  (TTD)  of  each  task  element  is  determined. 

Finally,  the  total  efficiency  (called  index  of  training 
device  effectiveness  in  the  TEEM  model)  of  a  training  device 
is  determined  by  the  following  formula: 

TC  +  PS  +  PS  +  SK  4  TTD 
3  2 

2 

A  detailed  description  of  the  TRAINVICE  procedure  is 
provided  in  Appendix  G.  A  worksheet  which  an  be  used  to 
summarize  the  results  of  the  TRAINVICE  procedure  is  provided 
in  Table  6-1.  Table  6-2  describes  guidelines  for  entering 
these  summary  results  into  the  SDT. 
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Table  6-1.  TRAINVICE  Summary  Worksheet  (TVICE) 


6.1.3  Estimate  Training  Method  Efficiency 
OVERVIEW 

The  same  general  procedures  and  algorithms  used  to  assess 
media  efficiency  can  be  used  to  assess  the  efficiency  of 
training  methods.  The  major  difference  between  the 
application  of  the  procedures  to  methods  rather  than  media 
is  that  a  different  set  of  varibles  must  be  used  to  assess 
the  match  between  methods  and  tasks. 

PROCEDURE 

An  overview  of  the  procedures  for  assessing  the  efficiency 
of  training  methods  is  presented  in  Figure  6-5.  More 
details  on  the  steps  in  this  procedure  are  provided  in  the 
sections  which  follow. 

o  Rate  Methods  on  Variables 

Table  6-3  lists  the  instructional  methods  which  are 
typically  used  in  the  Army.  Any  one  course  is  likely  to  use 
some  subset  of  these  methods.  The  methods  used  in  each 
course  must  be  rated  on  set  of  variables  which  assess  the 
match  between  the  methods  and  tasks.  A  candidate  listing  of 
such  variables  is  presented  in  Table  6-4.^  The  user  should 
select  five  to  twenty  of  these  variables.  The  variables 
selected  should  oe  those  which  are  most  likely  to 
discriminate  between  the  methods  included  in  the  course. 


3  These  are  the  variables  used  by  Jorgensen  (1979)  in  the 
TEEM  project  to  assess  the  efficiency  of  training  methods. 


Table  6-3.  Method  Alternatives. 


AT  Audio  Tape 

C  Conference/Lecture 

CAI  Computer  Assisted  Instruction 

CS  Case  Study 

D  Demonstration 

DF  Dual  Flight  Hours  {only  aviator  courses) 

(do  not  include  in  ICH  computations) 

El  Hardware  Performance  Examination 

E2  Nonhardware  Performance  Examination 

E3  Nonhardware  Performance  Examination 

EL  Elective  (in-house  only,  except  for  CGSC) 

F  Film 

GS  Guest  Speaker 

IS  Independent  Study 

NCI  Non-contact  Instruction  with  an  Instructor 
Available  in  Classroom 
NC2  Non-contact  Instruction  without  an 

Instructor  Available 

PEI  Hardware  Oriented  (hands-on)  Practical 

Application 

PE2  Nonhardware  Oriented  (non-classroom) 

Practical  Application 
PE3  Classroom  Practical  Application 

PI  Programmed  Instruction  (using  programmed  text) 

PM  Printed  Materials 

QC  Besseler  Cue  See 

S  Seminar 

SF  Solo  Flight  Hour s  (only  aviator  courses) 

(do  not  include  in  ICH  computations; 

SI  Simulation  Instruction 

3?  Self-Paced  Instruction 

ST  Slide  Tape 

TV  Television 

WC1  Instructor  Led  Work  Group 

WC2  Student  Led  Work  Group 


DA  Pam  57C-553  Staffing  Guile 
for  U.S.  Army  Service  Sc.cccls 
and  TRADCC  clr  351-12  Formas 
for  Programs  of  Instruction 


sources  : 


Table  6-4. 


Method -Re  1, 


Method  Variables 


STIMULI  CHARACTERISTICS 

Medium  of  Stimuli  Presentation 

1.  Visual  Cues  -  Signals  received  through  the  sense  of  sight. 

2.  Audio  Cues  -  Signals  received  through  the  sense  of  hearing. 

3.  Tactile  Cues  -  Signals  received  through  the  sense  of  touch,  including 
sensations  related  to  texture,  size,  shape,  or  vibration  of  the  skin. 

4.  External  Stimulus  Motivation  Cues  -  The  sensations  felt  by  a  person 
when  he  is  moved  by  some  outside  force  in  such  a  way  that  his  body 
experiences  roll,  pitch,  yaw,  heave,  sway  and/or  surge. 

5.  Internal  Stimulus  Motion  Cues  -  The  sensations  felt  by  a  person  when 
he  moves  his  arm,  leg,  fingers,  etc. 

6.  Olfactile  Cues  -  Signals  received  through  the  sense  of  smell. 

7.  Gustatile  Cues  -  Signals  received  through  the  sense  of  taste. 

Number  of  Channels  or  Sources 

8.  Limited  -  A  small  number  of  sources,  channels,  or  instruments  through 
which  stimuli  are  presented  to  the  trainee. 

9.  Unlimi  '  -  A  multiple  number  of  sources,  channels,  or  instruments 
throi  which  stimuli  are  presented  to  the  trainee. 

Stimuli  Pacing 

10.  Self-paced  -  Stimuli  are  only  presented  upon  the  trainee's  request. 

11.  Forced-pace  -  Stimuli  are  presented  at  some  predetermined  rate, 
frequency  of  change,  etc. 

Stimuli  Distribution 

12.  Individual  -  All  information  is  presented  or  displayed  directly  to 
one  individual  trainee. 

13.  Group  -  Information  is  presented  or  displayed  to  a  group  of  trainees, 
allowing  only  indirect  access  to  the  information  for  an  individual. 


*  Taken  from  Jorgensen  and  Hoffer  (1978) 
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Information  Structure-Related.  Variables*' 


STIMULI  CHARACTERISTICS 


Medium  of  Stimuli  Presentation 

1.  Visual  Cues  -  Signals  received  through  the  sense  of  sight. 

2.  Audio  Cues  -  Signals  received  through  the  sense  of  hearing. 

3.  Tactile  Cues  -  Signals  received  through  the  sense  of  touch,  including 
sensations  related  to  texture,  size,  shape,  or  vibration  of  the  skin. 

4.  External  Stimulus  Motion  Cues  -  The  sensations  felt  by  a  person  when 
he  is  moved  by  some  outside  force  in  such  a  way  that  his  body  exper¬ 
iences  roll,  pitch,  yaw,  heave,  sway  and/or  surge. 

5.  Internal  Stimulus  Motion  Cues  -  The  sensations  felt  by  a  person  when 
he  moves  his  arm,  leg,  fingers,  etc. 

6.  Olfactile  Cues  -  Signals  received  through  the  sense  of  smell. 

7.  Gus*~.atile  Cues  -  Signals  received  through  the  sense  of  taste. 


Stimuli  Presentation 

8.  Static  -  A  unitary  stimuli  situation,  i.e.,  stimuli  are  presented  to 
the  trainee  "all  at  once",  e.g.,  batch  presentations. 

9.  Dynamic-Ordered  -  A  sequential  stimuli  situation,  i.e.,  stimuli  are 
presented  to  the  trainee  sequentially  or  in  an  ordered  manner  over  time 

10.  Dynamic -Random  -  A  non-sequent ial  stimuli  situation,  i.e.,  stimuli 
are  presented  to  the  trainee  randomly  over  time. 


Stimuli  Presentation  Rate 

11.  Slow  Rate  -  A  slow  rate  or  speed  of  presentation  of  stimuli  to  the 
trainee,  allowing  the  trainee  a  long  or  maximum  stimulus  analysis  time. 

12.  Fast  Rate  -  A  fast  rate  cr  speed  of  ’presentation  of  stimuli  to  the 
trainee,  allowing  the  trainee  a  short  or  minimum  stimulus  analysis 
time . 


Frequency  of  Stimuli  Change 

13.  Infrequent  -  A  low  frequency  of  stimuli  change,  i.e.,  stimuli  presented 
to  the  trainee  change  from  one  to  another  infrequently. 

14.  Frequent  -  A  high  frequency  of  stimuli  change,  i.e.,  stimuli  presented 
to  the  trainee  change  from  one  to  another  frequently. 


* 


T  iken  from  Jorgensen  and  Hof  ter  (1978) 
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Number  of  Stimuli 

15.  Few  -  A  small  number  of  separate  streams  of  signa] s  that  occur 
independently  of  each  other,  but  require  simultaneous  consideration. 

16.  Multiple  -  A  large  number  of  separate  streams  of  signals  that  occur 
independently  of  each  other,  but  require  simultaneous  consideration. 

Stimuli  Format 

17.  Formatted  -  Formatted  or  organized  stimuli,  such  as  alphanumeric 
displays. 

18.  Unformatted  -  Unformatted  or  unorganized  stimuli,  such  as  raw  video 
displays. 

Patterning  of  Stimuli 

19.  Simple  -  Simple  or  common  patterns  of  stimuli,  such  as  the  repetition 
of  the  same  event  at  high  frequencies  of  occurrence. 

20.  Complex  -  Complex,  detailed,  or  circumstantial  patterns  of  stimuli, 
such  as  discrete  sequenced  patterns  at  low  frequencies  of  occurrence. 


RESPONSE  CHARACTERISTICS 

Response  Mode  of  Implementation 

1.  Overt  Response  -  Verbal  -  A  response  which  the  trainee  expresses  in  an 
audible  (verbal)  manner,  such  as  a  verbal  short  answer  response  to  a 
question  having  a  limited  set  of  correct  answers,  a  conversational 
response,  or  a  verbal  decision  response. 

2.  Overt  Response  -  Written  -  A  response  which  the  trainee  expresses 
in  an  observable  (written)  manner,  such  as  a  free  style  written 
response,  a  written  multiple  choice  response,  or  a  written  fill-in- 
the-blank  response. 

3.  Overt  Response  -  Manipulative  Acts  -  A  response  which  the  trainee 
expresses  in  an  observable  (manipulative)  manner,  such  as  the  small 
movements  of  dials,  switches,  keys,  or  small  adjustments  to  instruments 
or  the  large  movements  of  levers,  wheels  or  use  of  hand  held  tools. 

4.  Overt  Response  -  Tracking  -  A  response  which  the  trainee  expresses 
in  an  observable  (tracking)  manner,  such  as  continuously  controlling 
a  constantly  changing  system,  e.g.,  steering  an  automobile. 

5.  Overt  Response  -  Procedural  Performance  -  A  response  which  the  trainee 
expresses  in  an  observable  (procedural  performance)  manner,  such  as 
performing  a  sequence  of  steps  in  a  procedure,  e.g.,  carrying  out  the 
items  on  the  checklist  for  preflighting  an  aircraft  or  turning  on  a 
radar  system. 
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6.  Covert  Response  -  A  response  which  the  trainee  creates  in  his  mind 
but  does  not  express  in  an  observable  or  audible  manner. 


Response  Implementation 

7.  Static  -  A  unitary  response  situation,  i.e.,  responses  are  made  by 
the  trainee  "all  at  once". 

8.  Dynamic-Ordered  -  A  sequential  response  situation,  i.e.,  responses 
are  made  by  the  trainee  sequentially  or  in  an  ordered  manner  over  time 

9.  Dynamic -Random  -  A  non-sequential  response  situation,  i.e.,  responses 
are  made  by  the  trainee  randomly  over  time. 


Required  Response  Rate 

10.  Slow  Rate  -  A  slow  rate  or 
which  allows  the  trainee  a 

11.  Fast  Rate  -  A  fast  rate  or 
which  allows  the  trainee  a 


speed  of  trainee  response,  i.e., 
long  or  maximum  response  time. 

speed  of  trainee  response,  i.e., 
short  or  minimum  response  time. 


a  rate 

a  rate 


Response  Frequency 

12.  Infrequent  -  A  low  frequency  of  response  change,  i.e.,  responses  made 
by  the  trainee  change  infrequently. 

13.  Frequent  -  A  high  frequency  of  response  change,  i.e.,  responses  made 
by  the  trainee  change  frequently. 

Number  of  Required  Responses 

14.  Few  -  A  small  number  of  responses  that  the  trainee  is  required  to 
express,  each  independent  of  the  other. 

15.  Multiple  -  A  large  number  of  responses  that  the  trainee  is  required 
to  express,  each  independent  of  the  other. 


Response  Format 

16.  Formatted  -  Formatted  responses,  e.g.,  responses  to  a  programmed  text. 

17.  Unformatted  -  Unformatted  responses,  e.g.,  responses  made  by  team 
members  during  an  emergency  situation. 

Patterning  of  Responses 

18.  Simple  -  Simple  or  common  patterns  of  responses,  such  as  repetition  of 
the  same  response  at  high  frequencies  of  occurrence. 

19.  Complex  -  Complex,  detailed,  or  circumstantial  patterns  of  responses, 
such  as  discrete  sequenced  patterns  of  responses  at  low  frequencies  of 
occurrence. 
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INFORMATION  FEEDBACK  LOGIC 

Medium  of  Feedback  Presentation 

1.  Visual  -  Feedback  presented  visually  by  means  of  a  display,  it  may 
be  coded  and  transmitted  visually  to  the  trainee. 

2.  Aural  -  Feedback  presented  aurally  by  means  of  a  display  to  the 
trainee . 

3.  Written  Form  -  Feedback  presented  to  the  trainee  in  written  form. 

4.  Face-to-Face  Communication  -  Feedback  presented  by  direct  verbal 
means  to  the  trainee. 

5.  Indirect  Communication  -  Feedback  presented  by  indirect  verbal  means, 
such  as  by  intercom,  telephone,  or  radio  link. 

6.  Tactile  -  Feedback  presented  to  the  trainee  through  the  sense  of  touch, 
including  sensations  related  to  texture,  shape,  size,  or  vibration 

of  the  skin. 

7.  Kinesthetic  -  Feedback  presented  to  the  trainee  by  either  internal  or 
external  bodily  movement,  such  as  reaching,  grasping,  tilting,  etc. 

8.  Olfactile  -  Feedback  presented  to  the  trainee  through  the  sense  of  smell. 

9.  Gustatile  -  Feedback  presented  to  the  trainee  through  the  sense  of  taste. 

Time  Schedule  for  Feedback 

10.  Pre-F  Interval  -  Immediate  -  Feedback  provided  immediately  after  the 
trainee's  antecedent  response,  i.e.,  there  is  a  small  or  no  interval 

of  time  between  the  antecedent  response  and  the  feedback  for  that  response 

11.  Pre-F  Interval  -  Delayed  -  Feedback  provided  subsequently  after  the 
trainee's  antecedent  response,  i.e.,  there  is  an  interval  of  time  or 
delay  between  the  antecedent  response  and  the  feedback  for  that  response. 

12.  Post-F  Interval  -  Immediate  -  Presentation  of  the  next  stimulus 
immediately  after  the  occurrence  of  feedback  for  the  last  response, 
i.e.,  there  is  a  small  or  no  interval  of  time  between  the  occurrence 
of  feedback  for  the  last  response  and  the  presentation  of  the  next 
stimulus . 

13.  Post-F  Interval  -  Delayed  -  Presentation  of  the  next  stimulus  subsequently 
after  the  occurrence  of  feedback  for  the  last  response,  i.e.,  there  is 

an  interval  of  time  between  the  occurrence  of  feedback  for  the  last 
response  and  the  presentation  of  the  next  stimulus. 
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Feedback  Regularity 

14.  Regular  -  Feedback  provided  to  the  trainee  at  regular  intervals, 
such  as  after  every  or  every  other  response  or  at  established  or 
fixed  periods. 

15.  Irregular  -  Feedback  provided  to  the  trainee  at  variable  intervals, 
which  may  change  as  a  function  of  stage  of  training  or  level  of 
performance.  This  includes  the  provision  for  intermittent  presentations 
to  permit  probabilistic  schedules  for  reinforcement. 

Frequency  of  Feedback 

16.  Frequent  -  Any  feedback  given  to  the  trainee  that  is  provided  at  least 
for  20-30%  of  all  his  responses. 

17.  Infrequent  -  Any  feedback  given  to  the  trainee  that  is  provided  less 
than  for  20%  of  all  his  responses  or  is  not  provided  at  all. 

Feedback  Format 

18.  Formatted  -  Formatted  feedback,  e.g.,  alphanumeric  feedback  displays. 

19.  Unformatted  -  Unformatted  feedback,  e.g.,  raw  video  feedback  displays. 
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Response  Criterion 

14.  Determinate  -  The  response  criterion  has  a  fairly  explicit  or  implicit 
response  requirement  as  a  direct  reflection  of  the  training  or  system 
requirement,  and  makes  the  stimulus  (visual,  audio,  etc.)  possess  a 
certain  or  determinate  value  or  payoff  for  the  trainee. 

15.  Indeterminate  -  The  response  c:  iter ion  is  highly  dependent  on  the 
trainee's  interpretation  of  the  response  requirement,  and  makes  the 
stimulus  (visual,  audio,  etc.)  possess  an  uncertain  or  indeterminate 
value  or  payoff  for  the  trainee. 


RESPONSE  CP  '  ACTERISTICS 

Response  Mode  of  Implementation 

1.  Overt  Response  -  Verbal  -  A  response  which  the  trainee  expresses  in 
an  audible  (verbal)  manner,  such  as  a  verbal  short  answer  response  to 
a  question  having  a  limited  set  of  correct  answers,  a  conversational 
response,  or  a  verbal  decision  response. 

2.  Overt  Response  -  Written  -  A  response  which  the  trainee  expresses 
in  an  observable  (written)  manner,  such  as  a  free  style  written 
response,  a  written  multiple  choice  response,  or  a  written  fill-in- 
the-blank  response. 

3.  Overt  Response  -  Manipulative  Acts  -  A  response  which  the  trainee 
expresses  in  an  observable  (manipulative)  manner,  such  as  the  small 
movements  of  dials,  switches,  keys,  or  small  adjustments  to  instruments 
or  the  large  movements  of  levers,  wheels  or  use  of  hand  held  tools. 

4.  Overt  Response  -  Tracking  -  A  response  which  the  trainee  expresses 
in  an  observable  (tracking)  manner,  such  as  continuously  controlling 
a  constantly  changing  system,  e.g.,  steering  an  automobile. 

5.  Overt  Response  -  Procedural  Performance  -  A  response  which  the  trainee 
expresses  in  an  observable  (procedural  performance)  manner,  such  as 
performing  a  sequence  of  steps  in  a  procedure,  e.g.,  carrying  out  the 
items  on  the  checklist  for  preflighting  an  aircraft  or  turning  on  a 
radar  system. 

6.  Covert  Response  -  A  response  which  the  trainee  creates  in  his  mind 
but  does  not  express  in  an  observable  or  audible  manner. 
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Response  Pacing 

7.  Self-paced  -  Responses  are  made  by  the  trainee  at  his  own  rate. 

8.  Forced-pace  -  Responses  are  made  by  the  trainee  at  some  predetermined 
rate  or  frequency. 

Response  Distribution 

9.  Individual  -  All  responses  are  expressed  by  one  individual  trainee. 

10.  Group  -  Responses  are  expressed  by  a  group  of  trainees,  allowing 
only  indirect  responses  for  an  individual. 

Response  Determinancy 

11.  Determinant  -  A  required  response  which  can  be  specified  in  advance 
of  the  stimuli  that  calls  the  response  out,  i.e.,  the  response  is 
preprogrammed  or  determined  before  the  operation  begins. 

12.  Indeterminant  -  A  required  response  which  cannot  be  specified  in 
advance  of  the  stimuli  that  calls  the  response  out,  i.'e.,  the  response 
is  highly  dependent  on  the  immediate  stimuli  situation  and  cannot  be 
predicted  in  advance,  hence  is  indeterminate. 


INFORMATION  FEEDBACK  LOGIC 

Medium  of  Feedback  Presentation 

1.  Visual  -  Feedback  presented  visually  by  means  of  a  display,  it  may  be 
coded  and  transmitted  visually  to  the  trainee. 

2.  Aural  -  Feedback  presented  aurally  by  means  of  a  display  to  the  trainee. 

3.  Written  Form  -  Feedback  presented  to  the  trainee  in  written  form. 

4.  Face-to-Face  Communication  -  Feedback  presented  by  direct  verbal  means 
to  the  trainee. 

5.  Indirect  Communication  -  Feedback  presented  by  indirect  verbal  means, 
such  as  by  intercom,  telephone,  or  radio  link. 

6.  Tactile  -  Feedback  presented  to  the  trainee  through  the  sense  of  touch, 
including  sensations  related  to  texture,  shape,  size,  or  vibration  of 
the  skin. 

7.  Kinesthetic  -  Feedback  presented  to  the  trainee  by  either  internal  or 
external  bodily  movement,  such  as  reaching,  grasping,  tilting,  etc. 

8.  Olf actile  -  Feedback  presented  to  the  trainee  through  the  sense  of  smell 

9.  Gustatile  -  Feedback  presented  to  the  trainee  through  the  sense  of  taste 
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Source  of  Feedback 

10.  Intrinsic  F  -  Information  or  cues  built  into  the  system  from  which 
the  trainee  interprets  feedback  information. 

11.  Extrinsic  F  -  Information  or  cues  not  inherent  in  the  trainee  action 
or  system  operations  but  is  supplied  by  an  external  source. 

Feedback  Pacing 

12.  Self-paced  -  Feedback  presented  to  the  trainee  only  at  his  request. 

13.  Forced-pace  -  Feedback  presented  to  the  trainee  at  some  predetermined 
rate,  frequency,  etc. 

Feedback  Distribution 

14.  Individual  -  Feedback  is  presented  to  one  individual  trainee. 

15.  Group  -  Feedback  is  presented  to  a  group  of  trainees,  allowing  only 
indirect  access  for  an  individual. 


Table  6-4  Functional  Context  Variables  * 


ROLE  OF  ELEMENT  -  The  social  function  performed  by  the  trainee  within 
the  system's  operational  context. 

1.  Supervisory  -  The  trainee's  function  is  unequal  to  functions  being 
performed  by  other  individuals;  the  role  is  basically  one  of  over¬ 
seeing  or  directing. 

2.  Team  Performance  -  The  trainee's  function  is  equal  to  functions  being 
performed  by  other  individuals;  the  function  is  basically  one  of 
teamwork  and  cooperation. 

3.  Individual  Performance  -  The  trainee's  function  is  one  of  performing 
alone,  usually  without  supervision  or  team  assistance. 

FUNCTION  PERFORMED  IN  ROLE  -  The  primary  actions  performed  within  each  role. 

4.  Mental  -  An  action  occurring  or  experienced  in  the  trainee's  mind,  as 
contrasted  with  overt  physical  activity. 

5.  Physical  -  An  overt  bodily  action  performed  by  the  trainee. 

6.  Perceptual  -  An  action  by  the  trainee  involving  perception  or  observation 

7.  Communicative  -  An  action  by  the  trainee  in  which  he  transmits  either 
a  written  or  verbal  message. 

STABILITY  OF  FUNCTION  -  The  function's  state,  quality,  or  degree  of  being 
constant  over  time. 

8.  Unstable  -  The  trainee's  function  is  not  constant  or  regular,  it  is 
characterized  by  continual  change  and  fluctuation. 

9.  Stable  -  The  trainee's  function  has  little  change  or  fluctuation  over 
time. 

PHYSICAL  CONTEXT  -  The  significance  of  the  physical  environment  upon 
performance  of  the  task. 

10.  Low  Impact  -  The  physical  environment  has  little  or  no  significant 
impact  on  performance  of  the  task. 

11.  High  Impact  -  The  physical  environment  has  a  large  or  significant 
impact  on  performance  of  the  task. 

PSYCHOLOGICAL  IMPACT  -  The  significance  of  the  psychological  environment 
upon  performance  of  the  task. 

12.  Low  Impact  -  The  psychological  environment  has  little  or  no  significant 
impact  on  performance  of  the  task. 

13.  High  Impact  -  The  psychological  environment  has  a  large  or  significant 
impact  on  performance  of  the  task. 

*  Taken  from  Jorgensen  and  Hoffer  (1978) 


Once  the  variables  have  been  selected,  each  method  must  be 
rated  in  terms  of  whether  or  not  the  variable  applies  to 
that  method.  Table  6-5  contains  a  worksheet  which  can  be 
used  to  make  these  ratings.  The  entries  in  the  worksheet 
would  be  one  if  the  variable  applies  to  the  method  and  zero 
if  it  does  not. 

o  Rate  Tasks  on  Variables 

Each  of  the  tasks  to  be  trained  must  be  rated  on  the  same 
variables  used  to  assess  training  methods.  Table  6-6 
contains  a  worksheet  which  can  be  used  to  make  these 
ratings.  The  entries  in  the  worksheet  should  be  one  if  the 
variable  applies  to  the  method  and  zero  if  it  does  not. 

o  Determine  Method-Task  Match 

The  match  between  a  task  and  an  instructional  method  is 
determined  by  counting  the  number  of  times  the  same  variable 
is  rated  as  applying  to  both  the  method  and  the  media  (see 
Table  6-7  )  . 

o  Determine  Maximum  Possible  Match  Between  Each  Task 

The  maximum  possible  match  for  a  task  is  determined  by 
counting  the  number  of  variables  which  were  rated  as 
applying  to  a  task  (see  Table  6-6). 

o  Calculate  Efficiency  of  Each  Method 

Total  efficiency  for  a  method  is  determined  by  dividing  the 
method-task  match  score  by  the  maximum  possible  match  (see 
the  worksheet  listed  in  Table  6-7)  to  produce  an  efficiency 


Table  6-7.  Worksheet  for  Calculating  Method  Efficiency  (METHEFF) 


score  for  each  method-task  combination  and  then  summing 
across  tasks. 


o  Determine  Task  Criticality 

As  an  added  feature,  each  task  can  be  weighted  by  its  task 
criticality  during  the  calculation  of  total  efficiency  for  a 
method  (see  Table  6-7).  Procedures  for  calculating  task 
criticality  are  described  in  Procedure  3.1. 

o  Sum  Across  Methods 

Total  efficiency  for  a  course  can  be  determined  by  summing 
across  the  efficiency  scores  for  each  method. 

6.2  ESTIMATE  TRAINING  EFFECTIVENESS 

OVERVIEW 

This  procedure  can  be  used  to  develop  a  rough  measure  of  the 
effectiveness  of  a  training  course  during  the  early  phases 
of  the  acquisition  process.  In  this  procedure,  subject 
matter  experts  are  used  to  estimate  the  percentage  of 
soldiers  who  will  pass  the  criterion  for  each  task.  More 
specifically,  a  group  of  subject  matter  experts  is  presented 
with  the  following  information  for  each  task:  (a)  the  target 
population  description  of  the  personnel  who  will  perform  the 
task,  (b)  a  description  of  the  task  including  its  associated 
conditions,  job  performance  standards,  and  general  skills 
and  knowledges,  (c)  a  description  of  the  training  program  or 
training  program  elements  (e.g.,  methods,  media)  that  will 
be  used  to  train  the  task,  (d)  the  criterion  which  must  be 
achieved  for  the  task  (if  different  from  the  job  performance 
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measure),  and  (e)  the  training  time  to  be  devoted  to  the 
task.  Each  SME  is  then  asked  to  estimate  the  expected 
percentage  of  soldiers  in  target  population  who  will  pass 
the  criterion  given  that  training  program.  A  total 
effectiveness  measure  for  a  course  can  be  constructed  by 
averaging  the  effectiveness  scores  associated  with  each 
task. 

PROCEDURE 

An  overview  of  the  procedure  for  estimating  effectiveness  is 
presented  in  Figure  6-6.  A  more  detailed  description  of  the 
steps  in  this  procedure  is  provided  in  the  sections  which 
follow: 

o  Select  Subject  Matter  Experts 

To  provide  usable  estimates  of  effectiveness,  a  subject 
matter  expert  must  (1)  have  at  least  two  years  experience  in 
performing  the  task  or  other  comparable  tasks  and  (2)  have 
at  least  two  years  experience  in  Army  training  developments. 
As  many  individuals  as  possible  who  meet  the  above  criteria 
should  be  selected  (admittedly,  this  number  will  not  be 
large). 

o  Complete  Rating  Forms 


Table  6-8  presents  a  rating  form  which  can  be  used  to  elicit 
the  estimates  of  effectiveness.  One  of  these  forms  should 
be  filled  out  for  each  task  in  the  training  program. 
Obtaining  the  information  to  put  in  the  forms  should  be 
relatively  straightforward  since  all  of  this  information 
should  have  been  developed  in  previous  ETES  procedures. 


SELECT 

SUBJECT 

MATTER 

EXPERTS 


CONSTRUCT 
OVERALL 
MEASURE  OF 
EFFECTIVENESS 


Table  6-8.  Form  for  Rating  Effectiveness. 


Task  Description 


Task  Number: 


Comparable  Task:. 


Conditions: 


Major  Skills  and  Knowledges: 


What  percentage  of  soldier  can  be  expected  to  pass  the  criterion  given  the  training  program? 


•  »■  %  , 
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Table  6-9  describes  the  ETES  procedures  where  each  item  in 
the  rating  form  are  generated. 

o  Present  Ratings  to  SMEs 

Each  subject  matter  expert  should  be  presented  with  the 
rating  forms  for  all  of  the  tasks  in  the  course.  Prior  to 
the  presentation  of  these  forms,  the  SME  should  be  given  the 
QPOI  (Procedure  3.6)  and  target  population  description 
(Procedure  3.4)  for  the  course.  Each  SME  should  be 
instructed  to  leave  the  form  blank  if  he/she  is  not  familiar 
with  the  task  or  the  comparable  existing  task. 


o  Construct  Overall  Measures  of  Effectiveness 

Table  6-10  provides  a  worksheet  which  can  be  used  to  develop 
an  overall  effectiveness  measure  for  a  course.  Training 
effectiveness  estimates  should  be  taken  from  the 
effectiveness  rating  form  and  entered  into  columns  (a)  and 
(b),  respectively.  Mean  scores  should  be  calculated  across 
SMEs  and  entered  into  column  (c).  The  total  of  these  mean 
scores  (item  d)  should  then  be  calculated  and  divided  by  the 
number  of  tasks  to  provide  an  overall  measure  of  course 
effectiveness. 


ELEMENT 


RELATED  ETES  PROCEDURE 


Task  Number  and  Title 

Comparable  Task 

Conditions,  Standards 

Skills  and  Knowledges 

Training  Methods  and 
Media 

Training  Times 
Course 
Setting 
Criterion 


2.2,  2.4 
2.2,  2.4 


2.2,  2.4 


3.3 


3.6 


3.6 

3.6 

3.2 

3.6 


Table  6*10.  Worksheet  for  Calculating  Overall  Effectiveness  Score  (EFF) 
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SECTION  7.0  EVALUATE  TRAINING  PROGRAM 


OVERVIEW 

This  procedure  provides  techniques  for  evaluating  the 
training  program  which  was  developed  in  the  earlier  ETES 
procedures . 

PROCEDURE 

This  procedure  consists  of  six  lower  level  procedures.  An 
overview  of  these  procedures  is  presented  in  Figure  7-1. 

In  the  first  procedures,  eight  f igures-of-mer i t  for 
evaluating  the  “goodness"  of  a  training  program  are 
developed.  In  the  second  procedure,  these  f igures-of-mer it 
are  used  to  identify  likely  problem  areas  in  the  training 
program.  In  the  third  procedure,  the  likely  causes  of  the 
problem  areas  are  identified  In  the  fourth  and  fifth 
procedures,  methods  are  provided  for  identifying  and 
evaluating  training  alternatives.  In  the  sixth  procedure, 
the  impact  of  the  system  changes  are  determined 

7.1  DEVELOP  FIGURES-OF-MERIT 

During  this  procedure  training  Figures-of-Mer i t  are 
developed  to  evaluate  the  training  courses  developed  in  the 
previous  ETES  procedures.  A  listing  of  the  Figures-of-Merit 
which  may  be  used  to  evaluate  the  training  course  estimates 
during  the  earliest  phases  of  the  acquisition  process  is 
presented  in  Table  7-1.  A  training  Figure-of-Merit  provides 
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•  COST 

•  TRAINING  EFFICIENCY 

•  TRAINING  EFFECTIVENESS 

•  CONGRUENCE  WITH  TRAINING  PROGRAM  GUIDELINES 

•  CONGRUENCE  WITH  PROGRAM  REQUIREMENTS 

•  TRAINING  COMPLEXITY 

•  FEASIBILITY 

•  SUMMARY  EVALUATION  SCORE 
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Figure  7-1.  Overview  Diagram:  Evaluate  Training  Program  (7.0) 


Table  7-1.  Potential  Training  Figures  of  Merit 


1.  Cost  -  the  cost  of  the  training  program  or  training  program 
element  as  determined  in  ETES. 

2.  Training  Efficiency  -  a  measure  of  how  well  the  training 
program  matches  the  task  requirements  of  the  system. 

Training  efficiency  is  determined  by  comparing  the 
characteristics  of  the  tasks  to  be  trained  with  the 
characteristics  of  the  training  program  element. 

3.  Training  Effectiveness  -  training  effectiveness  is  measured 
by  assessing  the  proficiency  of  the  soldiers  who  experience 
the  training  program  element. 

4.  Congruence  with  Training  Development  Guidelines  -  a  measure 
of  the  degree  to  which  the  training  program  has  been  con¬ 
structed  in  accordance  with  recommended  or  specified  principles 
of  Army  instructional  system's  development. 

5.  Congruence  with  Program  Requirements  -  a  measure  of  how  well 
the  training  program  matches  requirements  outlined  in  require¬ 
ments  documents  (e.g.,  JMSNS,  LOA,  TDLOA)  or  in  request  for 
proposals.  This  measure  is  not  applicable  unless  meaningful 
statements  of  training  requirements  have  been  developed. 

6.  Training  Complexity  -  a  measure  of  the  training  complexity 

of  a  particular  system  or  system  element.  Training  complexity 
is  determined  by  summing  the  training  man-days  associated  for 
each  course  associated  with  the  system. 

7.  Feasibility  -  a  composite  measure  assessing  (a)  the  technical 
risk  associated  with  development  of  the  training  program  and 
(b)  the  likelihood  that  the  training  program  can  be  developed 
within  existing  resource,  cost,  and  schedule  constraints. 

8.  Summary  Evaluation  Score  -  a  summary  measure  combining  the 
three  key  training  f igures-of-merit  —  cost,  effectiveness, 
and  complexity. 


a  summary  measure  of  the  "goodness"  of  a  training  program  or 
course.  In  most  cases,  the  training  Figures-of-Merit  will  be 
developed  to  summarize  the  "goodness"  of  course.  In  some 
cases,  however,  they  may  be  applied  to  the  elements  of  a 
course.  Table  7-2  lists  the  training  program  or  course 
elements  which  are  likely  to  be  evaluated  during  the  early 
phases  of  the  acquisition  process  and  describes  which  of  the 
figures-of-  merit  are  most  appropriate  for  evaluating  each 
course  element. 

The  training  f igures-of-mer it  can  be  used  to  evaluate  the 
training-related  impacts  of  changes  to  non-training  system 
elements  such  as  changes  in  manpower  and  personnel 
assignments  and  requirements,  changes  in  system  hardware/ 
design,  and  the  overall  system  schedule  and  budget.  Table 
7-2  indicates  which  training  f igures-of-mer i t  can  be  used  to 
assess  the  impacts  of  each  of  these  non-training  system 
elements.  A  more  detailed  discussion  of  procedures  for  using 
training  f igures-of-mer it  to  assess  changes  in  overall 
system  design  is  presented  under  Procedure  7.6. 

Of  course,  the  individual  training  f igures-of-mer it  are  not 
independent  of  one  another.  Figure  7-2  presents  an  overview 
of  the  casual  relationships  among  the  training  figures  of 
merit.  A  more  detailed  discussion  of  these  casual 
relationships  is  presented  under  Procedure  7.3. 

Procedures  for  estimating  training  cost,  efficiency,  and 
effectiveness  were  presented  under  Procedures  5.0,  6.1,  and 
6.2,  respectively.  During  this  procedure,  procedures  are 
provided  for  estimating  the  other  Figures-of-Merit  listed  in 
Table  7-1. 
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Table  7-2.  System  Elements  and  Applicable  Figures  of  Merit. 


System  Element 

Cost 

Efficiency 

Effectiveness 

Congruence  with 

Training  Development 
Guidelines 

Congruence  with 

Program  Requirements 

Training  Complexity 

Feasibility 

Summary  Evaluation 
Score 

Training  Settings 

• 

• 

• 

• 

• 

• 

• 

• 

Methods 

• 

• 

• 

• 

• 

• 

• 

• 

Media 

• 

• 

• 

• 

• 

• 

• 

• 

Course  Sequencing 
(between  course) 

• 

• 

• 

• 

• 

• 

Training  Times 

• 

• 

• 

• 

• 

• 

Course  Scheduling 
(within  course), 

Frequency  and  Locations 

• 

• 

• 

• 

• 

Usage  Rates  for 

Training  Resources 

• 

• 

• 

• 

Assignment  of  Tasks  to 
MOS,  Duty  Position,  and 
Skill  Level 

• 

• 

• 

• 

MOS/Selection 

Requirements 

• 

• 

• 

• 

Hardware/Software  Design 

• 

• 

• 

• 

System  Schedule 
and  Budget 

• 

• 

• 

The  user  should  select  which  f igur es- of-mer i t  will  be 
applied  to  each  training  program  element  as  part  of: 
development  of  the  ETES  study  plan,  (see  Section  1.6), 

7.1.1  Assess  Congruence  with  Training  Development 
Guidelines 

OVERVIEW 

Several  documents  have  been  produced  to  guide  Army  training 
developers  in  developing  training  programs  for  emerging 
systems.  These  documents  represent  the  collective  wisdom 
and  expertise  of  the  training  development  community.  One 
can  assume  that  training  programs  which  have  been  developed 
in  accordance  with  these  guidelines  are  likely  to  be  more 
effective  than  training  programs  which  are  not.  This 
f igure-of-mer it  provides  a  measure  of  the  congruence  between 
the  procedures  actually  used  to  develop  the  training  program 
element  and  the  procedures  prescribed  in  existing 
guidelines.  This  f igure-of-merit  can  play  an  important  role 
in  assessing  the  training  products  developed  by  outside 

contractors.  Obviously,  if  this  measure  is  to  have  any 
credibility,  it  should  not  be  assessed  by  the  same 
organization  which  developed  the  training  program. 

PROCEDURE 

An  overview  of  the  procedure  for  measuring  the  congruence 
between  the  actual  procedures  used  to  develop  the  training 
program  element  and  the  prescribed  procedures  is  presented 
in  Figure  7-3.  First,  existing  documents  containing 

guidance  for  developing  the  particular  type  of  training 
program  elements  under  investigation  are  identified.  A 


prel lminary 


list  of 


examining  the  sources 


such  sources  can  be  generated 
listed  in  Table  2-3  and  the  list 


by 

of 


references  provided  at  the  end  of  this  guide.  Additional 


guidance  is  provided  in  the  Army  ISD  job  aids  developed  by 
Schulz  and  Farrell  (1980). 


Once  the  sources  have  been  identified,  a  checklist  must  be 
developed  specifying  the  observable  features  that  the 
training  program  element  would  display  if  it  was  developed 
in  accordance  with  the  guidelines  specified  in  the  sources. 
An  example  checklist  is  provided  in  Table  7-3. 


Once  the  checklist  has  been  developed,  each  program  element 
must  be  rated  as  to  whether  it  displays  the  characteristic 
required  in  each  item  on  the  checklist.  Following  this,  the 
importance  of  each  item  on  the  checklist  should  be  assessed 
using  the  following  scale: 


1.  Not  at  all  Important 

2.  Somewhat  Important 

3.  Moderately  Important 

4.  Important 

5.  Extremely  Important 


Completion  of  the  checklist  and  the  importance  ratings 
should  be  done  by  training  analyst  who  have  at  least  three 
years  experience  in  Army  training  developments.  Importance 
ratings  should  be  averaged  across  rates.  An  item  should  be 
rated  as  meeting  existing  guidelines  if  a  majority  of  the 
analysts  rate  the  item  as  meeting  requirements. 


Once  the  importance  of  the  items  has  been  determined,  a 
general  measure  of  the  congruence  between  actual  and 


Assessing  Congruence  Between  Actual  and  Prescribed  Guidelines 
or  Adequacy  of  Task  Statements 


TOTAL  MATCHS 


prescribed  training  development  procedures  can  be  developed 
using  the  following  algorithm: 


CT  D 


where  CT  D  is  the  general  congruence  measure,  1^  .is  the 
importance  of  the  kth  checklist  item;  equals  1  if  there 
is  a  match  between  the  actual  and  prescribed  practice  for 
the  kth  checklist  item  and  0  otherwise;  and  k  is  the  number 
of  checklist  items. 

Table  7-4  displays  a  worksheet  which  can  be  used  to  support 
the  development  of  the  training  congruence  Figure-of-Merit. 


7.1.2  Assess  Congruence  with  Program  Requirements 

It  is  possible  that  requirements  relating  to  the  training 
program  may  be  included  in  system  requirements  documents 
(e.g.,  JMSNS,  LOA).  Such  requirements  may  also  be  included 
in  the  request  for  proposals  (RFPs)  associated  with  the 
system.  One  measure  of  the  "goodness"  of  the  training 
program  is  the  extent  to  which  it  meets  these  requirements. 

This  f  igure-of-mer  it  can  play  an  important  role  in 
evaluating  contractor  developed  training  program  elements. 


It  should  be  noted  that  this  measure  is  less 
the  organization  setting  the  requirements  i 
organization  producing  the  training  program 


meaningful 
s  also  the 
element. 


when 

same 


Table  7-4.  Worksheet  for  Measuring  Congruence  with  Training  Development  Guidelines  (CONDEV) 


PROCEDURE 

The  first  step  is  to  identify  training-related  requirements 
by  examining  existing  system  requirements  documents  (JSMSN, 
LOA,  ROC,  TDLOA,  TDR)  and/or  contractor  Request  for 
Proposals.  Second,  a  checklist  must  be  developed  specifying 
the  observable  components  of  each  requirement.  Third,  each 
training  program  element  must  be  rated  on  the  extent  to 
which  it  meets  the  specified  requirements.  Fourth,  the 
importance  of  each  requirement  should  be  assessed  using  the 
following  scale. 

1  =  Not  At  All  Important 

2  =  Somewhat  Important 

3  =  Moderately  Important 

4  =  Important 

5  =  Extremely  Important 

Congruence  with  requiremeats  can  then  be  determined  using 
the  following  algorithm: 


CR  = 


Xk  Rk 


where  CR  is  the  general  congruence  measure?  1^  is  importance 
of  the  kth  requirement,  R^  equals  1  if  the  training  program 
meets  the  requirement  and  0  if  it  does  not*,  and  K  is  the 
number  of  requirements. 

A  generic  checklist  is  provided  in  Table  7-5.  A  worksheet 
which  can  be  used  to  support  the  development  of  this  figure- 
of-merit  is  displayed  in  Table  7-6.  Completion  of  the 
checklist  and  the  importance  ratings  should  be  done  by 
training  analysts  who  have  at  least  three  years  experience 
in  Army  training  developments.  Importance  ratings  should  be 
averaged  across  raters.  A  item  should  be  rated  as  meeting 


Generic  Checklist  for  Assessing  Congruence  with  Program  Requirements 


Table  7-6.  Worksheet  for  Measuring  Congruence  with  Program  Requirements  (CONPROG) 


Total 


existing  guidelines  if  a  majority  of  the  analysts  rate  the 
item  as  meeting  requirements. 


7.1.3  Assess  Training  Complexity 
OVERVIEW 

During  the  early  phases  of  the  acquisition  process,  it  is 
important  to  identify  the  system  elements  which,  generate 
excessive  or  complex  training  requirements.  By  constructing 
a  measure  of  training  complexity  of  each  system  element 
training  developers  can  help  materiel  developers  in 
identifying  potential  design  problems.  A  relatively  single 
measure  of  training  complexity  which  can  be  used  during  the 
early  phases  of  the  acquisition  process  is  training  time 
(course  length) . 

PROCEDURE 

Training  complexity  is  initially  determined  for  each  course 
associated  with  the  system.  A  summary  complexity  measure 
for  each  MOS  is  determined  by  summing  the  length  of  the 
courses  associated  with  the  MOS.  The  training  complexity  of 
a  system  may  be  determined  by  summing  the  length  of  the 
courses  for  all  MOSs  required  to  operate  or  maintain  each  of 
the  component  subsystems. 

The  training  complexity  measures  for  each  MOS  can  be 
examined  to  identify  the  system  areas  with  excessive 
requirements  (see  procedure  7.2  for  more  guidance  on 
procedures  for  identifying  problem  areas).  A  worksheet 
which  can  be  used  to  summarize  the  training  complexity 
measure  is  provided  in  Table  7-7. 
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Course _ I  MOS 1  Length  I  Course  I  MOS  I  Length  I  Course  MOS  Length 


TOTAL 


7.1.4  Assess  Feasibility 


OVERVIEW 

One  important  factor  to  consider  in  assessing  the  "goodness" 
of  training  program  is  its  feasibility  -  i.e.,  the  estimated 
likelihood  that  the  training  program  element  can  be 

developed  within  existing  technical,  cost,  resource,  and 
schedule  constraints.  The  feasibility  of  a  training  program 
consists  of  four  components: 

(1)  Technical  Feasibility  -  the  likelihood  than  any 
new  or  modified  technologies  in  the  training 
program  can  be  developed  given  the  current  state- 
of-the-art  in  training  technology.  If  existing 
technologies  are  employed,  this  factor  is  not 
relevant. 

(2)  Cost  Feasibility  -  the  likelihood  that  a  training 

program  or  training  program  element  can  be 

developed  and  maintained  within  the  existing 
budget  allotted  to  training  development  and 
operation. 

(3)  Resource  Feasibility  -  the  likelihood  that  a 

training  program  can  be  developed  and  implemented 
given  the  availability  of  critical  training 
resources  (e.g.,  instructors). 

(4)  Schedule  Feasibility  -  the  likelihood  that  a 

training  program  can  be  developed  in  accordance 
with  required  system  schedule  milestones. 


PROCEDURE 


An  overview  of  the  procedures  for  assessing  the  feasibility 
of  a  training  program  or  set  of  training  program  elements  is 
provided  in  Figure  7-4. 

First,  the  training  program  elements  are  rated  on  the 
feasibility  scales  listed  in  Table  7-8.  Second,  weights  are 
developed  to  reflect  the  user's  perception  of  the  importance 
of  each  of  the  scales.  (The  sum  of  the  weights  assigned 
should  equal  1.0).  Third,  an  overall  feasibility  score  is 
determined  using  the  following  algorithm. 


F  = 


Si 


where  F  is  the  overall  feasibility  score,  W  is  the  weight 
attached  to  the  ith  scale,  S  is  the  score  on  the  ith  scale, 
and  N  on  the  number  of  scale  items.  A  worksheet  to  support 
the  construction  of  the  feasibility  score  is  provided  in 
Table  7-9. 

The  feasibility  ratings  should  be  obtained  from  training 
analyses  who  have  at  least  three  years  experience  in  Army 
training  developments.  Ratings  should  be  averaged  across 
raters. 

7.1.5  Construct  Summary  Evaluation  Score 
OVERVIEW 

During  this  procedure,  a  summary  evaluation  score  is 
constructed  which  aggregates  scores  on  the  three  most 
critical  training  f  igures-of -mer i t :  cost,  effectiveness, 

and  training  complexity.  To  accomplish  this,  scores  for  the 
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Figure  7-4  Overview  Diagram  Assess  Feasibility  of 
Training  Program  Elements 
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Table  7  8.  Worksheet  for  Assessing  Feasibility  Scales  (FEASCAL) 
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(4)  Likelihood  that  training  program 
element  can  be  developed  in 
accordance  with  required  system 
schedule  milestones 


New  system  on  these  three  f  igures-of -mer it  are  divided  by 
the  Predecessor  score  on  the  same  f igures-of-meri t .  This 
converts  the  f igures-of-meri t  to  a  common  scale.  Weights 
are  then  assigned  to  each  FOM  and  the  weighted  scores  are 
then  summed  to  provide  a  summary  evaluation  score. 

PROCEDURE 

A  worksheet  which  can  be  used  to  construct  the  summary 
evaluation  score  is  presented  in  Table  7-10.  A  separate 
summary  evaluation  score  can  be  constructed  for  each  course 
or  training  program  element.  To  begin  the  procedure,  the 
user  should  enter  information  on  the  Predecessor  and  New 
system  cost,  effectiveness,  and  training  complexity 
(training  time)  into  the  appropriate  column  in  the 
worksheet.  Cost  values  are  produced  in  Procedure  5.1. 
Training  effectiveness  scores  are  produced  in  Procedure 
6.1.  Training  time  is  calculated  in  Procedure  3.5.  After 
these  values  have  been  entered  the  Predecessor  cost  and 
training  complexity  scores  should  be  divided  by  the  New 
system  scores  on  these  two  variables.  (See  Table  7-10). 
The  New  system  effectiveness  score  should  be  divided  by  the 
Predecessor  system  effectiveness  score. 1  Weights  should  be 
developed  for  each  of  the  three  f  igures-of-merit .  The 
weights  should  reflect  the  user's  estimate  of  the  individual 
importance  of  these  three  figures  of  merit.  The  weights 
should  dum  to  1.9  and  should  be  entered  into  the  appropriate 
columns  in  Table  7-10.  The  user  will  probably  want  to  use 
the  same  weight  for  each  course. 


^The  divisions  are  designed  to  produce  a  summary  evaluation 
score  where  high  scores  are  "good”  and  low  scores  are  "bad". 


Worksheet  for  Documenting  Summary  Evaluation  Score  (SUMEVAL) 


A  weighted  score  fore  ach  FOM  can  be  obtained  by  multiplying 
each  of  the  predecessor/New  system  ratios  by  the  weight  for 
the  FOM.  A  summary  evaluation  score  is  calculated  by 
summing  the  weighted  scores  for  the  three  FOMs. 

7.2  IDENTIFY  LIKELY  PROBLEM  AREAS 

OVERVIEW 

During  this  procedure,  training  problem  areas  are 
identified.  A  problem  area  is  defined  as  an  MOS  with  a  high 
score  on  the  three  f igures-of -me r it  which  provide  the  most 
critical  diagnostic  information  on  training  for  New 
systems.  These  f  igures-of-merit  are  total  training  cost, 
effectiveness,  and  training  complexity  (training  time). 

In  ETES,  problem  areas  are  identified  in  a  two  step  process. 
First,  all  areas  where  the  f igure-of-mer i t  for  the  New 
system  exceed  the  f igures-of-merit  for  the  Predecessor 
System  (that  is  where  the  New  system  does  not  fall  within 
the  footprint  of  the  Predecessor)  are  considered  to  be 
problem  areas.  These  are  the  areas  which  should  be  given 
primary  cons idera t ions  during  tradeoff  analyses  and 
identification  of  training  alternatives. 

Second,  the  areas  within  the  New  system  which  have  the 
highest  scores  on  the  figures  of  merit  are  identified.  These 
areas  represent  secondary  problem  areas  which  should  be 
examined  after  the  primary  areas  (i.e.,  the  areas  where  the 
New  system  does  not  fall  within  the  footprint  of  its 
predecessor)  have  been  examined. 


PROCEDURE 


An  overview  of  the  steps  in  this  procedure  is  outlined  in 
Figure  7-5.  During  the  procedure,  both  primary  and 
secondary  problem  areas  are  identified.  A  primary  problem 
area  is  the  occupational  specialty  (and  associated 
equipments)  where  the  New  system  training  f igures-of-merit 
exceed  the  Predecessor  system  f igures-of-mer i t  (i.e,  where 
the  New  system  does  not  fit  within  the  footprint  of  the 
predecessor).  Three  f igures-of-merit  are  used  for  this 
Predecessor-New  system  comparison:  (1)  total  cost,  (2) 
effectiveness,  and  (3)  training  complexity  (training  time). 

Secondary  problem  areas  are  identified  by  rank  ordering  the 
MOSs  in  the  New  system  by  their  associated  f igure-of-merit 
scores.  This  process  identifies  the  five  MOSs,  with  the 
highest  scores  on  each  of  the  three  f igures-of-mer i t 
described  above. 

This  procedure  begins  by  collecting  the  scores  on  the  three 
f igures-of-mer i t  (total  cost,  effectiveness,  and  complexity) 
for  all  MOSs  in  the  Predecessor  and  New  system.  A  worksheet 
for  describing  this  data  is  presented  in  Table  7-11.  Any 
MOS  where  the  f igure-of-mer i t  scores  for  New  system  is  15% 
greater  (or  15%  less  in  the  case  of  cost)  than  the  Predeces¬ 
sor  system  may  be  considered  to  be  a  primary  problem  area. 

Secondary  problem  areas  are  identified  by  rank/ordering  the 
MOSs  within  the  New  system  by  the  three  f igures-of-mer it  The 
MOSs  with  the  five  highest  lowest  in  the  case  of  effective¬ 
ness  scores  on  each  f igure-of-mer i t  are  identified.  Table  7- 
12  displays  a  worksheet  for  conducting  this  analysis. 
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Figure  7-5  Overview  Diagram  Identifying  Problem  Areas  (7.2) 


Worksheet  for  Identifying  Primary  Problem  Areas  (TRAINPROB) 


7.3  IDENTIFY  LIKELY  CAUSES  OF  PROBLEM  AREAS 


OVERVIEW 

During  Procedure  7.2  problem  areas  (i,e.,  MOSs)  related  to 
three  f igures-of-merit  (cost,  effectiveness,  and  training 
complexity)  were  identified.  In  order  to  identify 
alternatives  for  ameliorating  these  problem  aresa,  the 
likely  causes  of  the  high  scores  in  these  three  figures  of 
merit  must  be  determined. 

Typically,  the  term  "cause"  of  a  training  problem  is  used  in 
two  distinct  ways  (see  Figure  7-6). 

First,  it  is  used  to  refer  to  courses,  equipments,  and  tasks 
which  are  associated  with  the  high  f  igure-of-mer i t  scores. 
In  this  sense,  task  x  or  equipment  y  can  be  said  to  be  the 
cause  of  the  high  training  f igure-of-mer i t .  For  instance, 
one  MOS  may  have  an  exceedingly  high  training  cost  and  this 
high  cost  can  be  traced  back  to  the  course  associated  with 
one  particular  equipment  subsystem. 

Second,  the  term  "cause"  of  training  f  igure-of-mer i t  can  be 
used  to  refer  to  the  underlying  variable  which  leads  to  high 
f igure-of-mer it  scores.  For  example,  the  large  number  of 
students  to  be  trained  can  be  identified  as  the  "cause"  of 
high  training  costs. 

The  first  type  of  casual  analysis  identifies  "where"  the 
problem  lies  while  the  second  types  of  casual  analysis 
provides  information  on  the  nature  of  the  problem.  The 
second  type  of  casual  analysis  is  more  difficult  since  it 


requires  the  identification  and  examination  of  complex 
interrelationships  among  variables. 

The  procedure  described  in  this  section  deals  with  both 
types  of  casual  analysis. 

PROCEDURE 

An  overview  of  the  steps  in  this  procedure  is  provided  in 
Figure  7-7.  First,  the  courses  within  each  problem  MOS  that 
are  the  major  contributors  to  the  high  figures-of-  merit  are 
identified.  For  cost  and  training  complexity  (i.e.,  course 
length)  this  is  relatively  straightforward  since  courses  are 
the  basic  units  for  aggregating  these  f igures-of-mer i t .  For 
training  effectiveness,  which  uses  task  as  the  basic  unit  of 
analysis,  this  is  accomplished  by  examining  the  tasks 
associated  with  each  course. 

Following  the  identification  of  problem  courses,  problem 
equipments  (subsystems)  are  identified  by  examining  the  task 
descriptions  associated  with  each  course.  These  same  task 
descriptions  are  used  to  identify  the  tasks  within  each 
course  which  are  the  major  contributors  to  high  cost,  low 
effectiveness,  or  high  training  complexity. 

Once  the  problem  tasks  have  been  determined,  the  casual 
factors  likely  to  produce  the  high  f igures-of-meri t  are 
identified.  This  is  accomplished  by  first  examining  the 
casual  model  underlying  each  f igure-of-merit.  These  casual 
models  are  presented  in  Figures  7-8,  7-9  and  7-10.  To 

determine  likely  casual  factors,  the  user  should  start  with 
the  f igure-of-merit  and  move  backwards  in  the  casual 
structure  (from  right  to  left  in  Figures  7-8,  7-9  and  7- 

10).  Examining  the  values  of  the  variables  in  the  casual 
structure  and  noting  where  "excessively  high"  values  occur. 
To  identify  an  excessively  high  score,  the  user  should 
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Figure  7-8.  Overview  of  Causal  Structure  Underlying  Training  Cost. 
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Figure  7-9  Overview  Of  Causal  Structure  Underlying  Training  Complexity  (Training  Time) 
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Overview  Of  Causal  Structure  Underlying  Early  Measure  Of  Effectiveness 


compare  the  value  of  the  New  system  on  that  variable  to  the 
value  of  the  Predecessor  (or  BCS)  system  on  that  variable.  A 
difference  of  15%  or  greater  can  be  considered  excessive. 
Variables  with  excessively  high  difference  scors  which  are 
not  preceded  in  the  casual  structure  by  other  variables  with 
high  difference  scores  can  be  considered  potential  causes  of 
the  high  f ig ur e-of -me r i t .  Alternatives  which  may  reduce  the 
values  of  these  potential  causes  are  identified  in  Procedure 
7.4  A  worksheet  which  can  be  used  to  document  the 
identification  of  causes  is  presented  in  Table  7-13. 

7.4  IDENTIFY  ALTERNATIVES 

OVERVIEW 

During  Procedure  7.2  potential  training  problem  areas  were 
identified.  During  Procedure  7.3  likely  sources  of  these 
problems  were  identified  and  prioritized.  During  this 
Procedure,  alternatives  for  the  tra i ni ng-r elat ed  sources  are 
identified.  In  addition,  non-training  related  sources  of 
training  problems  (e.g.,  hardware/software  design)  are 
highlighted  and  brought  to  the  attention  of  the  Program 
Management  Office. 

PROCEDURE 

An  overview  of  the  procedure  for  identifying  training  alter¬ 
natives  is  presented  in  Figure  7-11.  First  the  sources  of 
training  problems  are  sorted  into  two  categories:  training- 
related  sources  and  non-training  related  sources. 
Alternatives  are  not  identified  for  the  training  related 
sources  since  these  are  beyond  the  purview  of  the  training 
analyst.  However,  the  non-training  related  sources  are 
presented  to  the  program  office  for  review. 


(a)  Problem  Area 


are 


The  training-related  sources 
identify  the  relevant  training 


examined  further  to 


program  element  associated 


with  the  alternative.  During  the  early  phases  of  the 
acquisition  process,  nine  major  categories  of  training 


alternatives  may  be  examined  (1)  settings,  (2)  methods,  (3) 
media,  (4)  course  sequencing  (between  courses),  (5) 


scheduling  of  modules  within  courses,  (6)  the  amount  of 
time  spent  in  training  tasks  within  a  course,  (7)  course 
frequency,  (8)  course  location,  and  (9)  usage  rates  for 
training  resources. 


The  general  alternatives  for  training  settings,  methods,  and 
media  are  well  defined.  These  alternatives  are  presented  in 
Tables  7-14,  7-15,  and  7-16.  The  user  may  determine  which 
method  or  media  alternative  is  likely  to  reduce  the  training 
problem  by  constructing  training  efficiency  scores  for  each 
alternative  (see  Procedure  6.1). 


The  alternatives  for  the  other  categories  of  training 
program  element  are  less  well-defined.  Users  can  obtain 
guidance  for  selecting  alternatives  in  these  areas  by 
examining  the  procedures  that  were  used  to  originally  create 
these  training  program  elements.  A  list  of  the  functions 
associated  with  the  score  categories  of  training  programs 
elements  is  provided  in  Table  7-17.  After  potential 
alternatives  have  been  identified,  the  feasibility  of  these 
alternatives  must  be  assessed  using  the  procedures  listed 
under  Procedure  7.1.4.  Subsequently,  the  alternatives  must 
be  prioritized,  taking  into  account  (a)  their  feasibility, 
(b)  the  priority  of  problem  source  with  which  they  are 
associated  (as  determined  in  7.2),  and  (c)  the  cost  of 
rerunning  the  appropriate  ETF.S  steps  to  assess  the 
alternatives  in  Procedure  7.5.  The  user  must  take  care  to 
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Table  7-14.  Training  Setting  Alternatives 


Institutional  Training  Settings  Skill  Level 

Basic  Training  (BT)  1 

Advanced  Individual  Training  (AIT)  1 

One  Station  Unit  Training  (OSUT)  1 

Primary  NCO  Course  (PNCOC)  2 

Basic  NCO  Course  (BNCOC)  3 

Advanced  NCO  Course  (ANCOC)  4 

Senior  NCO  Course  (SNCOC)  5 

Primary  Technical  Training  Course  (PTC)  2 

Basic  Technical  Training  Course  (BTC)  3 

US  Sergeants  Major  Academy  (USASMA)  — 

Support  School  (SPT  SCH) 

Primary  Leadership  Course  (PLC)  2 


Unit  Training  Settings 

Supervised  On-the-Job  Training  (SOJT) 

Self-Study 

Scheduled  Training 

Training  Extension  Courses  (TEC) 

Army  Correspondence  Course  Program  (ACCP) 


Table  7-15.  Training  Method  Alternatives 


AT  Audio  Tape 

C  Conference/Lecture 

CAI  Computer  Assisted  Instruction 

CS  Case  Study 

D  Demonstration 

DF  Dual  Flight  Hours  (only  aviator  courses) 

(do  not  include  in  ICH  computations) 

El  Hardware  Performance  Examination 

E2  Nonhardware  Performance  Examination 

E3  Nonhardware  Performance  Examination 

EL  Elective  (in-house  only,  except  for  C.GSC) 

F  Film 

GS  Guest  Speaker 

IS  Independent  Study 

NCI  Non-contact  Instruction  with  an  Instructor 
Available  in  Classroom 
NC2  Non-contact  Instruction  without  an 

Instructor  Available 

PEI  Hardware  Oriented  (hands-on)  Practical 

Application 

PE2  Nonhardware  Oriented  (non-classroom) 

Practical  Application 
PE3  Classroom  Practical  Application 

PI  Programmed  Instruction  (using  programmed  text) 

PM  Printed  Materials 

QC  Besseler  Cue  See 

S  Seminar 

SF  Solo  Flight  Hours  (only  aviator  courses) 

(do  not  include  in  ICH  computations) 

SI  Simulation  Instruction 

SP  Self-Paced  Instruction 

ST  Slide  Tape 

TV  Television 

WC1  Instructor  Led  Work  Group 

WC2  Student  Led  Work  Group 


Sources:  DA  Para  570-558  Staffing  Guide 
for  U.3.  Army  Service  Schools 
and  TRADOC  Cir  351-12  Format 
for  Programs  of  Instruction 
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Table  7-16.  Training  Media  Alternatives. 


Media  Category 

Print  Instructions  w/o  Feedback 

Print  Instructions  with  Oral  Feedback 

Print  Instruction  -  with  Written  Feedback 

Print  -  Reference  Material 

Image  Projection  -  Still 

Passive  Audio 

Active  Audio 

Audiovisual  -  Still  with  Feedback 
Audiovisual  -  Still  without  Feedback 
Audiovisual  -  Motion  with  Feedback 
Audiovisual  -  Motion  without  Feedback 
Static  Display  without  Feedback 
Dynamic  Display  without  Feedback 
Physiological  Trainer  -  Internal 
Physiological  Trainer  -  Audio 
Physiological  Trainer  -  Visual 
Symbolic  Simulation  with  Feedback 
Trainer 
Simulator 

Operational  Equipment 
Operational  Equipment  with  Feedback 
Visual  Computer  -  Still  with  Feedback 
Visual  Computer  -  Still  without  Feedback 
Visual  Computer  -  Motion  with  Feedback 
Visual  Computer  -  Motion  without  Feedback 
Audiovisual  Computer  -  Still  with  Feedback 
Audiovisual  Computer  -  Still  without  Feedback 
Audiovisual  Computer  -  Motion  with  Feedback 
Audiovisual  Computer  -  Motion  without  Feedback 
Instructor 
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Table  7-17.  Training  Project  Elements 


Training  Program  Element  Function 

Settings  3.2 
Methods  3.6 
Media  3.6 
Course  Scheduling  (Within  course)  3.5 
Training  Time  3.6 
Course  Frequency  4.1 
Course  Location  4.1 
Training  Resource  Usage  Rates  4.1 
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7.5  EVALUATE  ALTERNATIVES 

OVERVIEW 

During  this  procedure  the  training  alternatives  selected  for 
analysis  in  the  previous  step  are  evaluated  by  reapplying 
selected  ETES  procedures  and  sensitivity  analysis  are 
conducted  of  key  parameters. 

PROCEDURE 

Procedures  for  evaluating  alternatives  arid  conducting 
sensitivity  analyses  are  presented  in  the  sections  which 
f ol low . 

7.5.1  Evaluate  Alternatives 

An  overview  of  the  procedure  for  evaluating  alternatives 
presented  in  Figure  7-12.  First,  the  Training  Estimation 
Aids  and  Procedures  which  must  be  reapplied  to  assess  each 
alternative  must  be  identified.  This  is  accomplished  by 
examining  Table  7-17  which  lists  the  ETES  procedures  which 
must  be  reapplied  to  reflect  changes  or  alternatives  in  nine 
major  categories  of  training  program  elements. 


Once  the  ETES  Procedures  which  must  be  reapplied  have  been 


5 
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identified,  the  Evaluative  Technology  Procedures  which  must 
be  reapplied  are  determined.  To  accomplish  this,  the 
figures-  of-merit  that  will  be  used  to  evaluate  the 
alternative  must  be  identified. 

Once  all  of  the  required  ETES  Procedures  have  been 
identified,  they  must  be  reapplied.  As  the  steps  are 
applied,  the  worksheets  should  be  complete  to  document  the 
results  of  each  reapplication  of  an  ETES  procedure. 

7.5.2  Conduct  Sensitivity  Analyses 

Sensitivity  analyses  can  be  conducted  to  determine  how 
changes  in  key  variables  impact  the  results  of  the  training 
requirements  analysis.  Sensitivity  analyses  should  be 
conducted  for  variables  for  which  (a)  there  is  concern  about 
the  accuracy  of  the  values  assigned  to  the  variables,  or  (b) 
there  are  several  strong  competing  alternatives  which  may 
significantly  change  the  value  of  the  variable. 

The  procedure  for  conducting  sensitivity  analysis  is 
outlined  in  Figure  7-13. 

First,  the  variables  on  which  the  sensitivity  analyses  will 
be  conducted  are  determined.  Sensitivity  analyses  are 
typically  conducted  for  two  types  of  variables.  First, 
sensitivity  analyses  are  conducted  for  variables  where  there 
is  concern  about  the  accuracy  of  the  values  assigned  to  the 
variable.  For  instance,  a  questionable  value  has  been 
assigned  to  a  cost  element  for  a  course  and  the  user  is 
interested  in  seeing  how  changes  in  this  acquisition  cost 
would  impact  total  training  cost.  Second,  sensitivity 
analyses  are  conducted  for  variables  for  which  there  are  a 
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Figure  7  13  Overview  Diagram:  Conduct  Sensitivity  Analysis 
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number  of  strong  alternatives  which  may  change  the  vaJae  of 
the  variables.  For  instance,  if  the  user  be] j eves  that  the 
number  of  students  which  must  be  trained  could  range  from 
2000  to  3000  and  he/she  is  interested  in  determining  how 
training  resources  and  costs  will  be  impacted  by  these 
variations,  then  a  sensitivity  analysis  would  be  conducted 
on  this  variable. 

Once  the  sensitivity  variables  have  been  identified,  the 
user  must  identify  the  variables  he/she  will  use  to  assess 
the  sensitivity  impacts.  This  is  accomplished  by  examining 
Table  7-18  which  describes  the  major  variables  associated 
with  each  function  in  ETES.  Additional  information  is 
provided  in  the  causal  models  described  under  Procedure 
7.3.  In  general,  the  user  may  select  any  variable(s)  that 
are  associated  with  a  higher  numbered  function  (that  is  any 
variables  which  occur  later  in  the  causal  structure 
underlying  training  requirements).  Most  typically,  the  user 
will  examine  the  impact  of  changes  in  sensitivity  variables 
on  the  training  f  igures-of-mer  it .  All  procedures  listed  at. 
or  between  the  function  associated  with  the  sensitivity 
variable  must  be  reapplied. 

After  the  functions  to  be  reapplied  have  been  determined, 
the  user  must  identify  the  range  and  interval  values  to  be 
used  during  the  sensitivity  analyses.  The  range  of  values 
must  encompass  the  likely  alternatives  for  that  variable, 

Once  the  range  has  been  determined,  specific  values  at 
equally-spaced  intervals  within  the  range  must  be 
identified.  The  intervals  between  the  values  should  be  as 
large  as  possible  while  still  capturing  the  likely 
a lternat  ives . 
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Table  7-18  (Continued) 


Parameter 


ETES 


Number  of  Students 

Facilities  Requirements  per 
Student 

Number  of  Students 


Instructor  Contact  Hours 


Course  Length 


Media  Usage  Rate 


Number  of  Students 


4.4 

4.4 

4.5 

4.5 

4.5 

4.5 

4.6 


Number  of  Students 
Course  Length 
Number  of  Instructors 
Course  Frequency 
Reference/Unit  Training  Costs 
Criticality  Weights 


5.1 


5.1 


5.1 


5.1 


5.1 

6.1 


Procedure 

Determine  Facilities  Requirements 
Determine  Facilities  Requirements 

Determine  Training  Device  and 
Training  Equipment  Requirements 

Determine  Training  Device  and 
Training  Equipment  Requirements 

Determine  Training  Device  and 
Training  Equipment  Requirements 

Determine  Training  Device  and 
Training  Equipment  Requirements 

Determine  Requirements  for  Other 
Training  Resources 

Estimate  Individual  Course  Costs 

Estimate  Individual  Course  Costs 

Estimate  Individual  Course  Costs 

Estimate  Individual  Course  Costs 

Estimate  Individual  Course  Costs 

Estimate  Efficiency 


Once  the  range  and  values  have  been  determined,  the  selected 
ETES  functions  must  be  reapplied.  Table  7-19  presents  a 
worksheet  which  can  be  used  to  summarize  the  results  of 
sensitivity  analyses. 

7.6  EVALUATE  IMPACT  OF  SYSTEM  CHANGES 

OVERVIEW 

As  the  development  of  the  system  progresses  there  are  likely 
to  be  changes  to  non-MPT  system  elements  such  as  the 
hardware/sof twa re  design  and  manpower  requirements.  During 
this  subfunction,  the  impacts  of  these  changes  on  training 
are  determined. 

PROCEDURE 

There  are  two  steps  in  this  procedures.  First,  the  ETES 
procedures  which  must  be  reapplied  in  response  to  system 
changes  must  be  determined.  This  is  accomplished  by 
examining  Table  7-20  which  lists  the  functions  which  must  be 
reapplied  to  reflect  changes  in  non-MPT  system  elements  such 
as  the  hardware/software  design  or  manpower  requirements. 
Second,  the  identified  procedures  must  be  reapplied.  All 
changes  and  results  should  be  entered  into  the  ETES  audit 
trail  so  that  a  complete  record  of  the  system  development 
process  is  maintained. 


Table  7-19.  Worksheet  for  Describing  Results  of  Sensitivity  Analyses  (SENS). 


Table  7-20.  ETES  Procedures  Which  Must  Be  Reapplied 
In  Response  to  System  Changes 


System  Variable  Changed 

Design  Concepts 

Weapon  System  Requirements 

Organizational  &  Operational 
Concepts 

System  Functions 

Mission  Profile  &  Operational 
Mode  Summary 

Threat  and  Scenario 

System  Acquisition  Objectives 
and  Constraints 

System  Acquisition  Schedule 

MOS/Duty  Position  Assignments 

Skill  Level  Assignments 

Manpower  Requirements 


ETES  Functions  Which  Must 
Be  Reapplied _ 

1.6,  2.0  to  8.0 

1.1  to  1.6,  2.0  to  8.0 

1.1  to  1.6,  2.0  to  8.0 

1.1  to  1.6,  2.0  to  8.0 

1.1  to  1.6,  2.0  to  8.0 

1.2  to  1.6,  2.0  to  8.0 

1.1  to  1.6,  2.0  to  8.0 

1.1  to  1.6,  3.0  to  8.0 
2.3,  2.5,  3.0  to  8.0 
2.3,  2.5,  3.0  to  8.0 

4.2  to  4.6,  5.0,  6.0,  7.0,  8.0 
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SECTION  8.0  -  DEVELOP  INPUTS  TO  ACQUISITION  PROCESS 


OVERVIEW 

During  this  procedure,  a  training  development  schedule  is 
constructed  and  ETES-related  inputs  to  Army  acquisition 
processes  and  documents  are  specified. 

8.1  DEVELOP/MONITOR  TRAINING  DEVELOPMENT  SCHEDULE 

Procedures  for  developing  a  training-development  schedule 
are  provided  in  the  Automated  Planning  and  Scheduling 
Technique  (APST).  An  overview  of  APST  is  provided  in 
Appendix  C.  A  more  detailed  description  of  APST  is  provided 
in  the  User's  Guide:  Automated  Planning  and  Scheduling 
Technique . 

APST  can  be  used  to  assist  training  developers  in  describing 
and  monitoring  the  training  development  schedule  for 
developing  Army  weapon  systems.  APST  is  designed  to  be  used 
with  the  Visischedule  software,  which  is  an  automated 
program  for  describing,  monitoring  and  reporting  schedule 
information  and  for  conducting  critical  path  analyses  of 
schedule  events. 

8.2  DEVELOP  INPUTS  TO  ACQUISITION  PROCESSES  AND  DOCUMENTS 

OVERVIEW 

This  procedure  describes  how  the  products  of  the  other  ETES 
aids/procedures  can  be  used  to  support  Army  acquisition 
documents  and  processes. 
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Table  8-1  lists  the  documents  and  processes  which  have  the 
most  relevance  to  early  training  estimation.^ 

The  two  documents/processes  which  are  most  critical  to  early 
training  estimation  were  (1)  the  Outline  Individual  and 
Collective  Training  Plan  (OICTP)  and  (2)  Cost  and  Training 
Effectiveness  Analysis  (CTEA).  More  details  on  the 

relationships  between  ETES  procedures  and  the  Army  acquisi¬ 
tion  documents  processes  are  provided  in  the  sections 
8.2.11.  An  overview  of  the  relationship  between  ETES  and 
the  Army  Life  Cycle  System  Management  Model  ( LCSMM)  is 
provided  in  Section  8.2.12. 

8.2.1  Develop  Inputs  to  OICTP/ICTP 

The  OICTP  and  ICTP  are  plans  which  support  the  development 


and  implementation 

of 

new 

or 

revised  individual 

and 

collective  training 

programs 

at 

institutional  and 

unit 

levels.  The  OICTP 

and 

ICTP 

are 

the  major  resources 

and 

planning  documents 

for 

developing 

training  for  new 

Army 

systems.  An  approved  OICTP  is  sufficient  justification  to 
enter  manpower  and  funding  requirements  into  the  Army’s 
programming  and  budgeting  processes  for  inclusion  of  the 
TRADOC  Review  of  Manpower  (TRM).  As  specified  in  existing 
regulations  TRADOC  Reg.  351-9),  the  OICTP/ICTP  contains  a 
proposed  training  concept  and  training  strategy.  The 
concept  and  strategy  are  typically  generated  by  the 
proponent  school  responsible  for  the  New  system's  training 
requirements. 


1  Overviews  of  the  Army  acquisition  process  are  provided  in  DA 
Pam  11-25  and  in  Wagner  (1982). 
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The  OICTP/ICTP  is  an  evolving  document  that  increases  in 
specificity  (via  appropriate  updates)  as  the  system  under 
development  is  further  defined.  The  OICTP/ICTP  provides 
significant  feeder  data  for  the  Tentative  Qualitative  and 
Quantitative  Personnel  Requirements  Information  (TQQPRI), 
the  Training  Effectiveness  Analysis  (TEA),  and  the  New 
Equipment  Training  Plan  ( NETP ) .  After  LOA  approval,  the 
OICTP  can  be  used  by  the  contractor  to  facilitate  the 
identification  of  training  requirements  conducted  as  part  of 
the  logistic  support  analysis. 

The  OICTP/ICTP  is  intended  to  guide  the  development  of 
training  subsystem  requirements  and  to  provide  the  general 
framework  for  their  incorporation  into  the  existing  training 
base.  The  regulations  specifically  emphasize  the  intention 
to  use  the  proposed  training  concept  in  the  OICTP/ICTP  to 

"...identify  the  constraints  which  training  requirements  and 
resources  may  impose  on  the  design  of  the  materiel  system" 
(TRADOC  Reg  351-9,  pg .  5).  The  regulation  also  mandates  use 
of  the  OICTP/ICTP  as  the  vehicle  to  describe  "...  the 
integration  of  the  training  subsystem  into  the  development 
of  the  total  system  and  the  integration  of  the  developing 

system  into  ongoing  training  systems." 

TRADOC  Reg  351-9  further  stipulates  that  the  OICTP/ICTP  must 
incorporate  the  principles  of  Army  Training  1990  (AT  90) 
into  training  for  a  new  system,  for  both  institutional  and 

unit  training  and  at  all  skill  levels  for  the  MOSs  affected. 

The  OICTP/ICTP  is  intended  to  develop  and  describe  a 
systematic  and  feasible  strategy  for  training,  ranging  from 
the  development  of  "initial  qualification"  training  to  the 
"sustainment  of  the  proficiencies"  needed  for  the  successful 
fielding  and  operational  deployment  of  the  system  being 
acqu i r ed . 
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As  specified  in  existing  regulations,  the  OICTP/ICTP 
provides  information  on  the  training  required  to  integrate 
replacements  from  the  training  base  into  the  unit,  and  to 
qualify  personnel  for  higher  level  tasks  as  they  advance  in 
grade.  The  OICTP/ICTP  is  further  directed  to  provide 
information  on  the  identification,  quantification,  and  need 
for  training  devices,  simulator,  documentation/publications, 
training  aids,  support  facilities,  instructors,  costs,  and 
all  other  support  and  logistic  considerations  necessary  for 
the  implementation  and  test  of  the  proposed  training  plan. 

Table  8-2  provides  an  overview  of  the  elements  in  the 
OICTP/ICTP  and  describes  which  ETES  procedures  can  provide 
input  to  the  development  of  these  elements. 

The  TRADOC  Training  Effectiveness  Analysis  Handbook  produced 
by  TRANSANA  specifies  the  elements  which  must  be  included  in 
an  institutional  training  concept  that  is  part  of  the  OICTP. 
Most  information  elements  which  are  required  in  the 
institutional  training  concept  can  be  produced  by  applying 
ETES  procedures. 

Table  8-3  lists  the  information  elements  of  the 
institutional  concept  and  describe  which  ETES  procedure  can 
be  used  to  produce  these  elements.  A  more  detailed  descrip¬ 
tion  of  information  elements  in  the  training  concept  is 
provided  in  Appendix  H.  This  description  is  taken  directly 
from  the  TRADOC  Training  Effectiveness  Analysis  Handbook. 

8.2.2  Develop  Inputs  to  Cost  and  Training  Effectiveness 
Analysis  (CTEA) 

The  CTEA  process  is  potentially  the  most  critical  LCSMM 
process  related  to  early  training  estimation  since  it 
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Table  8-3.  Relationship  Between 
Concept  Elements  and 

Inst i t ut iona 1  Tr a  in ing 

ETES  Procedures- 

TRAINING  CONCEPT 

ETES 

ELEMENT 1 

PROCEDURE 

1.0 

Courses  Impacted 

3.6 

1.1 

Modification  or  New  Course 

3.6 

1.2 

Changes  to  Current  Courses 

3 . 6 

2.0 

Student  Load  per  Class  per  Year 

4.2 

2.1 

Average  Grade  of  Student 

— 

2.2 

Student  Source 

3.4 

3.0 

Class  Frequency  Per  Year 

4.] 

3.1 

Length  of  Class 

3.6 

3.2 

Fiscal  Year  of  Course  Start 

4.1 

4.0 

Instructor  Reqs.  Per  Course 

4.3 

5.0 

Support  Personnel  Requirements 

4.2 

6.0 

Expended  Equipment  Per  Class 

4.6  (partially) 

7.0 

Non-Expended  Equipment  Per  Class 

4 . 5 

8.0 

Exportable  Training  Used* 

Not  Covered  in 
Current  ETES 

8.1 

Exportable  Software* 

Not  Covered  in 
Current  ETES 

8.2 

Training  Terms 

3.6 

8.3 

Exportable  Hardware* 

Not  Covered  in 
Current:  ETES 

9.0 

Facility  Requirements 

4.4 

*  These  elements  refer  to  institutional  training  elements 
sent  to  units  in  the  field. 


requires  the  user  to  not  only  estimate  what  the  training 
program  will  look  like  (as  does  the  OICTP/ICTP),  but  also  to 
evaluate  these  training  programs  and  to  provide  training- 
related  input  into  the  overall  system  development  and 
evaluation  process. 


The  new  TRADOC  Reg  351-9,  governing  the  OICTP  process, 

contains  the  most  current  definition  of  the  CTEA.  It 

defines  CTEA  as  "  a  methodology  which  involves  documented 
investigation  of  the  comparative  effectiveness  and  costs  of 
alternative  training  systems  for  attaining  defined 

performance  objectives."  The  definition  further  specifies 
that  a  CTEA  can  focus  on  any  one  or  a  combination  of  the 

following : 


o 

o 

o 

o 

o 
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Regardless  of  the  specific  focus  of  the  CTEA,  TRADOC 
Regulation  351-9  stipulates  that  the  CTEA  should  include  an 
analysis  of  the  attainable  levels  of  proficiency  and  the 
costs  associated  with  each  alternative.  In  addition,  a  CTEA 
should  include  a  cost  effectiveness  trade-off  analysis  of 
the  feasible  alternatives.  This  regulation  further 


specifies  that  a  CTEA  must: 


o 

Insure  that 

training  development 

is  in 

i tiated 

early  in  the 

life  cycle  of  hardware 

systems 

and  is 

accompl i shed 

in  coordination 

with 

combat 

developments , 

o  Optimize  soldier-hardware  subsystem  interface, 

o  Insure  that  all  feasible  training  subsystem 
alternatives  are  considered, 


o  Optimize  soldier-training  subsystem  interface, 

o  Recommend  the  preferred  training  alternative  based 
on  cost  and  training  effectiveness,  and 

o  Provide  decision  makers  with  more  precise 
information  at  critical  points  in  the  acquisition 
process  concerning  the  total  system  (comprising 
the  training,  hardware,  and  other  subsystems). 


These  objectives  demonstrate  that  the  CTEA,  unlike  the  OICTP 
and  the  QQPRI/BOIP,  is  an  early  tra in i ng-re la t ed  document 
intended,  in  theory  at  least,  to  influence  the  hardware 
system  design.  This  is  a  very  significant  difference  from 
the  OICTP,  which  is,  by  regulation,  primarily  an  MPT 
planning  and  resource  document. 


The  steps  in  the  CTEA  process  and  the 
procedures  which  can  provide  input  to 
listed  in  Table  8-4. 


correspond i ng 
this  process 


ETES 

are 


LI 


STEPS  !N  THE  CTEA  PROCESS 


8.2.3  Develop  Inputs  to  Logistic  Support  Analysis 


As  defined  in  AR  700-127,  Logistics  Support  Analysis  (LSA) 
is  the  use  of  analytical  tools  and  models  to  (1)  develop  and 
evaluate  alternative  support  concepts,  (2)  project  manpower 
and  personnel  impacts,  (3)  perform  tradeoffs  between  system 
design  and  I LS  elements  and  tradeoffs  among  I LS  elements, 
integrated  support  planning  and  design,  and  (5)  measure  life 
cycle  impact  of  materiel  and  support  system  alternatives. 
The  outputs  of  LSA  are  used  to  (a)  influence  materiel  design 
or  selection,  (b)  develop  the  required  logistic  support 
system,  (c)  provide  the  manpower  and  logistics  analysis  for 
the  intergrated  program  summary,  and  (d)  record  the  LSA 
during  the  materiel  acquisition  phase  to  compare  it  with 
operational  use. 

Training  is  one  of  the  Integrated  Logistics  Support  (ILS) 
elements.  Hence,  all  of  the  analyses  requirements  required 
by  the  LSA  apply  to  training. 

One  of  the  key  components  of  LSA  is  the  Logistics  Support 
Analysis  Record  ( LSAR ) .  The  LSAR  sheets  provide  a  formal 
mechanism  for  describing  the  support  elements  of  a 
developing  system.  As  such,  LSAR  sheets,  like  the  ETES  SDT, 
can  be  reviewed  as  a  tool  for  describing  developing  support 
concepts.  Table  8-5  describes  the  LSAR  elements  which  are 
included  in  the  SDT  and  the  ETES  procedures  in  which  these 
elements  are  generated. 

8.2.4  Develop  Input  to  Operational  Testing 

As  defined  in  DA  Pam  11-25,  operational  testing  is  the 
testing  and  evaluation  of  materiel  systems  which  is 


accomplished  with  typical  user  operators,  crews,  or  units  in 
as  realistic  an  operational  environment  as  possible  to 
provide  data  to  estimate  the  military  utility,  operational 
effectiveness,  and  operational  suitability  (including 
compa tab il i ty ,  interoperabiltiy  ,  safety,  reliability, 
availability,  and  maintainability,  supportabili ty , 
operational  man  ( soldier )-machine  interface  and  training 
requirements)  of  new  systems.  Training  is,  and  should  be, 
one  of  the  key  elements  considered  during  OT.  The  Army 
Research  Institute  has  developed  a  procedure  called  HRTES 
(Human  Resource  Test  and  Evaluation  System)  which  is 
specifically  designed  to  provide  human  factors  and  training- 
related  input  to  OT  testing. 

An  overview  of  the  functions  performed  in  HRTES  is  presented 
in  Figure  8-1. 

Table  8-6  displays  the  relationships  between  ETES  functions 
and  HRTES  functions.  The  HRTES  system  index  assists 
analysts  in  identifying  comparable  existing  systems  which 
may  be  used  to  identify  system  mission  functions.  Systems 
mission  functions  are  then  identified  and  documented  in  a 
mission  chart.  The  term  "mission"  in  HRTES  is  equivalent  to 
the  concept  of  function  described  in  ETES  Function  1.0. 
Once  the  missions  (i.e.,  functions)  have  been  identified, 
system  performance  measures  for  these  functions  are 
identified.  HRTES  also  contains  procedures  for  evaluating 
the  results  of  operational  testing. 

The  HRTES  procedures  for  identifying  OT  training  issues 
(Human  Performance  Functions,  Human  Performance  Measures, 
Test  Conditions,  Human  Resource  Issues  and  Measures)  can  be 
used  to  identify  OT  test  requirements  within  the  total  ETES 
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Table  8-6.  Relationships  Between  HRTES  Function 
and  ETES  Function. 


HRTES  Function 


ETES  Function 


System  Index 

Mission  Charts 

System  Performance  Issues 

Human  Performance  Functions 

Human  Performance  Measures 

Test  Conditions 

Human  Resources  Issues  and 
Measures 

Operational  Test 
Evaluation  of  OT  Results 


1.4  Establish  Baseline  Comparison 
System 

1.1  Identify  Required  System 
Functions 

1.2  Identify  System  Performance 
Measures  and  Goals 

2.4  Develop  New  System  Task  List 


Diagnosis  of  Performance 
Inadequacies 


framework.  Direct  input  for  these  procedures  is  provided  by 
the  system  functions  (missions  in  HRTES  terminology)  and 
performance  measures  identified  in  Function  1.0.  An  overview 
of  HRTES  procedures  for  identifying  OT  test  items  and  issues 
is  provided  in  Appendix  I. 

8.2.5  Develop  Inputs  to  Training  Device  Requirements, 
Documents  and  Processes 

A  Training  Device  Requirement  (TDR)  is  a  document  which 
states  concisely  the  minimum  operational,  technical, 
logistical,  and  cost  information  necessary  for  development 
of  a  training  device.  A  TDR  is  required  if  the  funding 
thresholds  exceed  $2  million  in  RDTE  funds  and/or  S3  million 
for  any  fiscal  year  or  more  than  S15  million  for  the  5  year 
program  in  procurement  funds. 

Training  device  requirements  which  are  not  covered  by  TDRs 
are  specified  in  the  Training  Device  Letter  Requirement 
( TDLR  )  . 

The  Training  Device  Letter  of  Agreement  is  a  jointly 
prepared  and  authenticated  document  iri  which  the 
combat/training  developer  (TRADOC)  and  the  materiel 
developer  outline  the  basic  agreements  for  further 
investigation  of  a  potential  training  device  system. 

During  ETES  Procedure  3.7,  general  requirements  for  training 
devices  are  identified.  More  specifically,  the  tasks  which 
are  most  likely  to  require  training  via  devices  are 
identified.  This  information  provides  the  essential  input 
data  needed  to  initiate  a  Training  Development  Study 
(TDS).  A  TDS  is  a  cost  and  training  effectiveness  analysis 


conducted  to  support  the  development  of  training  devices. 
(See  TRADOC  Reg  351-9  for  a  description  of  TDS  procedures). 
In  addition,  the  resource  requirements  estimates  (e.g., 
number  of  students  to  be  trained)  made  during  Procedure  4.0, 
provide  input  to  the  estimation  of  cost  and  resource 
requirements  for  the  training  device.  The  TRAINVICE 

methodology  described  in  Procedure  6.1  and  in  Appendix  G 
provides  a  mechanism  for  calculating  the  training 
"efficiency"  of  an  emerging  training  device  concept. 

8.2.6  Develop  Inputs  to  New  Equipment  Training 

New  Equipment  Training  (NET)  provides  for  the  initial 
transfer  of  knowledge  from  contractor  and/or  materiel 
developer  to  tester/user  that  is  needed  for  operator 
maintenance  and  logistic  support  during  testing  and  initial 
introduction  of  new  materiel  into  the  Army. 

In  the  past,  training  developers  have  often  produced  initial 
resident  training  courses  by  modifying  the  NET  courses,  many 
of  which  were  developed  in  an  unsystematic  fashion.  Using 
the  ETES  procedures,  a  different  concept  of  course 
development  can  be  employed.  More  specifically,  using  ETES 
procedures,  an  early  description  of  the  system  training 
program  (that  is,  the  training  program  that  will  be  used 
when  the  system  is  fielded)  can  be  developed  first .  Task 
differences  and  target  population  differences  between  the 
NET  courses  and  the  system  training  program  can  then  be 
identified  u>ing  the  procedures  described  in  Procedures  2.0 
and  3.4.  NET  courses  can  be  generated  using  Procedure  3.6 
(Construct  Quas i-Program  of  Instruction).  Resource  and  cost 
estimates  can  then  be  generated  usiri  the  Procedures  4.0 
(Estimate  Training  Resources)  and  5.0  (Estimate  Training 


Costs).  This  strategy  has  an  obvious  advantage  in  that  it 
starts  with  the  end  objective  (system  training  program)  and 
then  identifies  the  intermediate  products  (NET  courses) 
needed  to  meet  those  objectives. 

8.2.7  Develop  Inputs  to  System  Requirements  Documents 

The  Justification  for  Major  System  New  Start  (JMSNS)  is  used 
to  identify  and  support  the  need  for  a  new  or  an  improved 
mission  capability.  The  JMSNS  is  required  only  for 

justifying  the  initation  of  a  new  major  acquisition  ($75 
million  or  more  RDTE  or  $300  million  or  more  in  acquisition 
costs).  If  an  ETES  analyses  is  conducted  during  mission 
area  analysis,  it  will  be  possible  to  develop  an  initial 
training  concept  which  may  be  included  in  the  JMSNS. 

The  LOA  is  a  requirement  document  jointly  prepared  by  the 
combat  developer  and  materiel  developer  which  outlines  the 
agreements  for  further  investigation  of  a  potential  materiel 
system  during  the  demonstration  and  validation  phase. 

Table  8-7  summarizes  the  relationships  between  ETES  products 
and  the  LOA. 

8.2.8  Develop  Inputs  to  Request  for  Proposal  (REP) 
Development/Proposal  Evaluation 

There  are  four  ways  in  which  ETES  related  products  can  be 
used  to  assist  in  the  development  and/or  evaluation  of  the 
proposals  which  are  generated  for  system  development. 
First,  the  results  of  ETES  can  be  used  to  specify  the 
minimally  acceptable  criteria  which  the  training  program 
which  is  developed  by  the  contractor  must  meet.  This 


Table  8-7.  ETES-LOA  Relationships. 


criteria  can  be  included  in  the  Request  For  Proposal  (RFP; 
which  is  generated  for  the  system. 

Second,  the  ETES  Evaluative  Technology  (Procedure  7.2)  can 
be  used  to  identify  likely  training  problem  areas  which  must 
be  addressed  in  detail  by  the  contractor.  Third,  the 
Baseline  Comparison  System  (BCS)  and  New  System  training 
programs  generated  by  ETES  provide  a  standard  with  which  to 
evaluate  the  training  programs  generated  by  contractors.  For 
example,  if  the  length  of  a  contractor  generated  course  is 
significantly  less  than  the  length  of  an  FTES  generated 
course,  the  training  developer  may  want  to  examine  the 

contractor  course  in  more  detail  to  determine  if  the 
contractor  is  underestimating  training  requirements  to  make 
their  design  concept  look  less  training  intensive. 

Fourth,  if  the  user  is  going  to  apply  ETES  or  some  portion 
of  ETES  and  wants  the  contractor  to  supply  the  input  data 

needed  to  feed  the  ETES  procedures,  ETES  input  data  require¬ 
ments  must  be  included  in  the  RFP.  Correspondingly,  if  the 
user  wants  the  contractor  to  apply  ETES  procedures,  ETES 
output  requirements  (at  the  procedural  step  level)  must  be 
included  in  the  RFP.  If  the  latter  approach  is  taken,  the 

usei  must  insure  that  ETES  input  data  from  existing  Army 
data  bases  is  included  in  the  government  furnished 

information  (GFI)  specified  in  the  RFP. 

8.2.9  Develop  Inputs  to  Personnel  Document. s/Processes 

The  Qualitative  and  Quantitative  Personnel  Requirements 
Information  (QQPRI)  describes  the  personnel  skills  required 
to  operate  and  support  a  specific  materiel  system;  tin 
recommended  placement  within  the  current,  revised,  r 


8-2  1 


Army  Military  Occupational  Specialties  (MOS),  including  a 
listing  of  duties  and  tasks;  new  or  revised  training 
requirements,  and  projected  annual  manpower  requirements. 
Table  8-8  displays  the  relationship  between  the  elements  of 
the  QQPRI  and  ETES  procedures. 

According  to  TRADOC  Reg  600-4,  Integrated  Personnel  Support 
(IPS)  is  the  process  by  which  personnel  considerations  are 
integrated  into  the  development  effort  for  a  materiel 
system.  Accomplishment  of  the  IPS  requires  that  each 
personnel  consideration  be  evaluated  in  terms  of  its  impact 
on  the  total  personnel  support  requirements  and  system 
design.  A  goal  of  this  evaluation  is  a  quantitative/ 
qualitative  statement  of  personnel  requirements  which  will 
insure  the  development  of  the  personnel  support  necessary  to 
achieve  the  desired  operational  capability  of  the  materiel. 
The  goals  of  the  IPS  roughly  parallel  the  goals  of  the  Cost 
and  Training  Effectiveness  Analysis  (CTEA)  which  is 
conducted  to  assess  training.  However,  the  IPS  has  a  strong 
emphasis  on  personnel  planning  and  in  this  sense  has 
objectives'  similar  to  the  Outline  Individual  and  Collective 
Training  Plan  (OICTP).  Table  8-9  lists  the  key  IPS  products 
which  are  generated  during  the  first  two  phases  of  the 
acquisition  process  and  the  relationships  between  these 
products  and  the  ETES  procedures. 

8.2.10  Develop  Inputs  to  System  Level  Documents  and 
Processes 

The  Concept  Formulation  Package  (CFP)  documents  the  studies 
conducted  during  the  exploration  of  alternative  systems 
concepts  conducted  during  the  first  phase  of  the 
acquisition.  The  CFP  consists  of  four  elements: 


Table  8-8.  ETES— QQPRI  Relationships. 


Training  Plan  8.1  Develop/monitor  training  development  schedule 


(1)  Cost  and  Operational  Effectiveness  Analysis  (COEA) 
provides  essential  information  on  the  cost  and 
effectiveness  of  materiel  system  alternatives  to 
permit  evaluation  and  decision  on  the  courses  of 
action  open  for  acquisition.  The  CTEA  is  one  of 
the  key  components  of  the  COEA. 

(2)  Tradeoff  Determinations  (TOD)  describe  the 
apparent  technical  feasibility  of  a  potential 
system,  including  technical  risks  associated  with 
each  approach,  estimated  RDTE,  and  procurement 
costs. 

(3)  Tradeoff  Analyses  (TOA)  determine  what  technical 
approach  offered  in  the  TOD  is  best. 

(4)  Best  Technical  Approach  consolidates  the  results 
of  the  TOD  and  TOA. 

The  CFP  is  included  in  the  Decision  Coordinating  Paper  (DCP) 
which  is  a  management  summary  document  to  support  the  Army 
System  Acquisition  Review  Council  (ASARC)  and  the  Defense 
System  Acquisition  Review  Council  ( DSARC ) .  ASARC/DSARC 
reviews  are  major  management  reviews  conducted  prior  to 
entry  into  successive  phases  of  the  materiel  acquisition 
process.  DSARCs  are  for  major  systems  requiring  Secretary 
of  Defense  approval.  ASARCs  are  for  major  systems  requiring 
Secretary  of  the  Army  approval. 

ETES  does  not  provide  direct  input  to  the  high  level 
acquisition  documents/processes  described  above.  However, 
ETES  does  provide  input  to  the  CTEA  (see  Section  8.2.1) 
which  is  in  turn  included  in  the  CFP  and  higher  level 
documents . 


8.2.11  Develop  Inputs  to  Mission  Area  Analysis/Soldier 
Analysis 

Mission  Area  Analysis  (MAA)  is  the  continuing  analyses  of 
mission  areas  which  is  conducted  to  identify  those  mission 
elements  for  which  existing  or  projected  capability  is 
deficient  and  to  identify  opportunities  to  enhance  the 
capability  to  achieve  and  sustain  combat  operations  through 
more  efi  ;tive  and  less  costly  methods  and  systems.  As  part 
of  MAA,  now  system  concepts  may  be  identified.  ETES  provides 
a  mechanism  for  evaluating  the  training  requirements  of 
these  early  concepts. 

Soldier  Analysis  provides  the  initial  mechanism  for 
consideration  of  soldier  factors  in  the  MAA.  It  requires 
MAA  proponents  to  identify,  prioritize,  and  evaluate  soldier 
factors  early  in  the  development  of  a  solution  to  a 
deficiency  and  provides  a  timely  and  quantified  basis  for 
further  studies  to  evaluate  the  effects  of  soldier  factors 
upon  solution  effectiveness.  Table  8-10  lists  the  products 
which  are  generated  by  Soldier  Analysis  and  the  ETES 
procedures  which  can  provide  input  to  the  generation  of 
these  products. 

8.2.12  Relationship  Between  ETES  and  LCSMM 

Figure  8-2  describes  the  LCSMM  events  which  are  directly 
related  to  ETES.  ETES  inputs  to  the  LCSMM  events  are 
described  in  Sections  8.2.1  to  8. 2. 1.1. 
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Figure  8-2  LCSMM  Events 
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Appendix  A:  Overview  of  Media  Selection  Program 


This  appendix  provides  a  brief  overview  of  the  Media 
Selection  Program.  A  more  detailed  description  is  provided 
in  the  User's  Guide;  Media  Selection  Program.  The  Media 
Selection  Program  is  an  automated  tool  for  (1)  assigning 
tasks  to  media  and  (2)  calculating  the  efficiency  and 
effectiveness  of  various  task-media  combinations.  The  Media 
Selection  Program  allows  users  to  assign  tasks  to  media  in  a 
manner  that  maximizes  overall  efficiency,  maximizes  overall 
"effectiveness,"  or  minimizes  overall  cost.l  In  addition, 
it  allows  users  to  assign  tasks  to  media  in  a  manner  that 
optimizes  various  combinations  of  these  variables,  including 
an  overall  "utility"  measure  which  combines  either  cost  and 
efficiency  or  cost  and  effectiveness. 

Efficiency,  in  the  Media  Selection  Program,  is  determined  by 
comparing  the  stimulus,  response,  and  feedback 
characteristic  of  the  tasks  to  the  stimulus,  response,  and 
feedback  characteristics  of  potential  media.  More 
specifically,  a  score  is  calculated  which  describes  the 
match  between  media  and  tasks  on  these  characteristics. 
Efficiency  for  each  task-media  combination  is  calculated  by 
dividing  this  score  by  the  maximum  match  that  may  be 


The  program  uses  a  relative  cost  and  not  actual  cost  to 
measure  the  potential  cost  requirements  of  media.  In 
addition,  the  program  does  not  use  a  measure  of 
"effectiveness"  that  fits  the  most  common  usage  of  that 
terms.  Rather  effectiveness  is  actually  efficiency  weighted 
by  task  criticality. 


achieved  for  the  task.  Total  efficiency  for  a  set  of  tasks 
is  the  sum  of  the  efficiency  score  for  individual  tasks. 

"Effectiveness"  is  calculated  by  weighting  the  efficiency  of 
each  task  by  a  task  criticality  score.  The  task  criticality 
score  is  a  user-defined  weighted  combination  of  the  eight 
task  factors  typically  used  in  selecting  tasks  for  training. 
These  eight  factors  are  task  frequency,  percent  members 
performing,  percent  time  performing,  task  delay  tolerance, 
consequences  of  inadequate  performance,  task  learning 
difficulty,  probability  of  deficient  performance,  and  time 
between  entry  and  performance. 

A  matrix  of  relative  cost  values  is  stored  in  the  program 
for  each  major  media  category.  In  addition,  a  built-in  set 
of  algorithms  is  used  to  produce  the  utility  measure  which 
combines  cost  with  either  effectiveness  or  efficiency. 

PROCEDURAL  OVERVIEW 

The  Media  Selection  Program  is  an  interactive  menu-driven 
system.  This  means  that  users  do  not  have  to  know  or  use  a 
computer  language  to  run  the  program.  Instead,  they  can  run 
through  the  program  by  selecting  options  from  a  series  of 
menus. 


c  This  concept  of  media  efficiency  was  originally 
developed  by  Jorgensen  (1978)  and  was  incorporated  into 
Training  Efficiency  Estimation  Model  (TEEM)  described  in 
Jorgensen,  Kubula,  and  Atlas  (1981). 


An  overview  of  the  procedures  for  using  the  Media  Selection 
Program  is  provided  in  Figure  A-l.  To  begin  the  procedures, 
users  must  rate  each  task  on  (1)  the  psychological  variables 
to  be  used  to  assess  the  match  between  tasks  and  media,  and 
(2)  the  variables  to  be  used  to  assess  task  criticality  (the 
latter  is  only  necessary  if  effectiveness  is  being 
calculated).  These  scores  must  be  then  entered  into  the 
ETES  data  base  management  system,  the  System  Description 
Technology.  When  this  is  completed,  users  must  enter  the 
Applications  Program  mode  in  the  System  Description 
Technology,  and  select  the  Media  Selection  Program.  Once  in 
the  Media  Selection  Program,  users  must  then  select  criteria 
to  be  used  in  making  media  assignments.  Seven  options  for 
selecting  criteria  are  provided:  (1)  efficiency,  (2) 
"effectiveness"  (3)  relative  cost,  (4)  cost  and  efficiency, 
(5)  cost  and  "effectiveness,"  (6)  a  utility  measure, 
combining  efficiency  and  cost,  and  (7)  a  utility  measure 
combining  "effectiveness"  and  cost. 

Once  the  criteria  have  been  selected,  users  must  select  the 
tasks  to  be  included  in  the  analysis  (only  tasks  already  in 
the  SDT  may  be  selected).  Typically,  the  tasks  for  single 
course  module  will  be  selected  for  each  analysis.  With  the 
analysis  criteria  identified,  the  psychological  variables  to 
be  used  in  calculating  the  match  between  tasks  and  media 
must  be  selected.  Users  may  select  from  12  variables 
assessing  stimuls  characteristics,  six  variables  assessing 
response  characteristics  and  four  variables  assessing 
feedback  characteristics.^ 


J  These  psychological  variables  were  taken  directly  from 
the  TEEM  Model  (Jorgeson,  Kubula,  and  Atlas;  1981) 


If  the  user  has  selected  a  set  of  criteria  involving 
effectiveness  (effectiveness,  cost  and  effectiveness,  or 
utility  with  effectiveness),  the  weights  for  the  component 
criticality  variables  must  be  entered  so  that  a  composite 
task  criticality  score  can  be  computed.  Eight  variables  may 
be  used  in  the  calculation  of  task  criticality:  (task 
frequency,  percent  member  performing,  percent  time 
performing,  task  delay  tolerance,  consequences  of  inadequate 
performance,  task  learning  difficulty,  probability  of 
deficient  performance  and  time  between  entry  and 
performance ) . 

At  this  point,  the  program  has  all  of  the  information  needed 
to  calculate  efficiency  and/or  effectiveness.  It  will  use 
this  information  to  calculate  the  relevant  measure  and  it 
will  display  the  results. 

If  the  user  has  selected  one  of  the  two  utility  measures 
(combining  either  cost  and  efficiency  or  cost  and 
effectiveness)  the  user  will  then  be  required  to  enter  the 
weights  to  be  used  in  the  computation  of  utility.  Following 
this,  the  user  must  select  the  objectives  and  constraints  to 
be  used  in  assigning  tasks  to  media.  A  listing  of  the 
possible  combinations  of  objectives  and  constraints  is 
displayed  in  Table  A-l.  Once  this  information  has  been  put 
into  the  computer,  the  program  will  optimally  assign  the 
tasks  to  training  media.  For  instance,  if  the  user  selected 
"maximum  effectiveness"  as  an  objective  and  "minimize  cost" 
as  a  constraint,  the  program  would  determine  the  assignment 
of  tasks  to  media  which  gives  the  highest  overall  score  on 
effectiveness  and  still  remains  under  a  user-specified  level 
of  overall  cost.  Once  the  initial  assignments  have  been 
examined,  the  user  can  examine  the  effects  of  several 


3SOD 

X 

3sauaAT-)oaj  J3 

-T-*  ..**TTT3R„ 

■ 

,^4Txian, 


5SOD 

DUB  AOU3T3T3J3 


SS3'I3AT103J  J3 


XouaxoT }}3 


pH 

03 

c 

XJ 

03 

O 

-C 

XJ 

XJ 

XJ 

u 

oj 

d) 

JZ 

XJ 

u 

03 

OJ 

JZ 

u 

0 

cr 

o 

in 

in 

• 

•H 

in 

rc 

in 

<u 

in 

<u 

XJ 

in 

c 

03 

a 

CJ 

U 

c 

> 

0) 

o> 

•H 

c 

> 

XJ 

oj 

•  H 

O 

O' 

XJ 

a> 

O 

Xj 

CO 

0) 

Xh 

•pH 

xi 

03 

Xj 

03 

<u 

Xh 

•H 

O 

'U 

pH 

a; 

03 

£ 

XJ 

d) 

0 

> 

O 

XJ 

OJ 

XJ 

n 

-H 

in 

c 

XJ 

j* 

03 

0J 

in 

•H 

03 

t: 

Xj 

XJ 

0) 

•H 

N 

o 

pH 

•*H 

0 

pH 

e 

a 

03 

•  H 

in 

X 

i 

Xj 

03 

Xt 

0 

£ 

a» 

in 

XJ 

in 

3 

C 

■ph 

0) 

0 

u 

pH 

0 

03 

XJ 

C 

0 

03 

XJ 

JZ 

xJ 

XJ 

03 

U 

-C 

di 

XJ 

xJ 

03 

x: 

d> 

a 

X! 

3 

cr 

in 

in 

T3 

■  pH 

0) 

xJ 

s 

03 

>1  S 

u 

<u 

XJ  • 

>1 

03 

i-H 

in 

nj  o 

in 

XJ  XJ 

oj 

0 

c 

XJ  pH 

0J 

03 

> 

x>  3 

-H 

ra  cr 

XJ 

x:  0) 

0 

XJ 

d; 

u 

Xj 

JZ  0 

U-l 

0 

dJ 

3  C 

in  03 

XJ 

x: 

in 

T3  xJ 

oj 

O 

XJ 

xj  in 

03 

oj  in 

<2J 

u  OJ 

u 

Qj  pH 

cr 

c 

<u  in 

dj 

cr  *h 

JS 

XJ 

in  xj 

•H  in 

x: 

O 

XJ 

03  O 

•H 

-H 

3 

■a  'H 

<V  03 

e 

e 

3 

0 

•pH 

o  XJ  . 

•a 

xj  X. 

<u 

T3  in 

e 

W  C  0 

-X  03  O 

d) 

in 

JS 

03  T3  xj 

XJ 

•u  4J  0 

N 

T3 

0) 

> 

o 

c 

N 

d) 

■pH 

•  -H 

e 

o 

•«x 

■  pH 

X 

UJ 

01 

<X4 

£ 

oj 

^H 

3 

n  or 

XJ 

2 

0 

XJ 

XJ 

XJ 

x< 

03 

d; 

JZ 

XJ 

XJ 

03 

x: 

QJ 

H 

u 

cr 

3 

in 

in 

T3 

oj 

■H 

>4 

XJ 

0 

• 

03 

c 

> 

w 

0) 

O 

0 

•pH 

c 

c 

O 

<y 

cu 

•H 

•pH 

cr  *X4 

O 

Xj 

•pH 

in 

d> 

IXJ 

■H 

XJ 

pH 

X 

o 

pH 

v 

a> 

03 

E 

Cu 

in 

Xj 

in 

i 

0 

.pj 

H 

d> 

XJ 

in 

in 

C 

in 

3 

d> 

dJ 

E 

c 

03 

C 

d> 

cr 

> 

0 

0 

XJ 

pH 

u 

03  -H 

0 

xj  03 

XJ 

0  3 

xj  cr 

£ 

a> 

>1 

jj 

XJ 

03  h 

-  pH 

x:  0 

^H 

XJ 

•  pJ 

C 

XJ 

-C  03 

3 

O  JZ 

s 

3  xj 

in 

XJ 

in 

in 

TJ  in 

d> 

d>  0) 

XJ 

XJ  pH 

03 

03 

O 

u  in 

H 

dJ  -pH 

cr 

c 

0)  XJ 

at 

cr  in 

-C 

c 

XJ 

in  o 

•pH 

J3 

pH 

XJ 

03  03 

•pH 

•H  XJ 

3 

•o  o 

a;  xj 

E 

E 

3 

Xj  xj 

■pH 

O  c  in 

V 

-U  03  0 

d> 

o 

E 

in  xj 

^  d)  xj 

d> 

in  n  O 

JZ 

03  -h 

XJ 

XJ  £  pH 

•h  ci> 

^H  X  > 

dJ 

pH  03  d) 

C 

03  E  pH 

cr 

•pH 

Xj  in  Xj 

in 

0  —l  0) 

in 

•  pH 

05 

xJ  10  Xj 

c  in  -h 

in 

dj  d)  o 

•H 

ECO) 

c  qj  a. 

jh: 

cr  >  w 

in 

T>  CJ> 
ai 

XJ  V)  * 
03  -h  >* 

*  o 

<v  >,  c 

CO  OJ 
(U  C  -H 

cr  ai  u 


« 3 

JZ  0> 

O  U 

a  cr 
in 

in 

"O  *h 

a) 

xj  in  . 
ro  w  > 
w  a>  o 
<d  c  c 
c  oj  oj 

<U  >  — < 

CP  — ♦  CJ 

J->  -H 

in  a  xj 

•ph  m  xj 
xj  0) 
03  xh 

•ph  q;  x» 
T5  0 
OJ  »H 
e  (ij  h 
xj 


in 

XJ 

in 

O 

pH 

03 

d) 

03 

Xj 

3 

c 

xj 

cr 

< 

d) 

o 

XJ  XJ 

d) 

pH 

in  xj 

.*  c 

XJ 

in  ot 

d3 

03 

•  pH 

XJ  X3 

Xj 

<u 

•  pH 

pH  n 

O 

pH  *H 

<D 

03  E 

a 

•H 

in 

X4  * 

\ 

0  03 

u 

E 

di 

XJ 

in 

c  in 

3 

QJ  -pH 

E 

03 

C  s 

cr  >i 

0 

I  -p*  4->  Xrf 

in  -h 

in  *h 

pH 

03  H 

03 

XJ 

3 

C  3 

cr 

<  * 

0J 

XJ 

XJ 

le 

in 

X 

j* 

c 

03 

in 

03 

u 

03 

•pH 

XJ 

X3 

XJ 

O 

•pH 

pH 

N 

o 

pH 

•pH 

d> 

03 

E 

a 

■  pH 

in 

Xj 

X 

i 

0 

03 

H 

E 

a> 

XJ 

« 

C 

in 

3 

OJ 

■  H 

E 

X 

C 

s 

cr 

> 

0 

in 

jj 

in 

rj  O  • 
•-  O  >* 
X3  O 

a>  ph  c 

£  03  a> 

XJ  *h 

o  o  a 


in  -h 

in  -h 

pH 

*»H 

03 

XJ 

3 

C  3 

cr 

<  * 

in 

in 

03 

c 

0> 

s 

d>  > 

a>  >h 

N 

N  XJ 

•pH  Jj 

•  pH  •  H 

E  O 

E  pH 

•pH  Qj 

•-H  -H 

X  Xh 

X  H 

.*0  xj 

03  3 

E  0) 

e  = 

A-6 

.  .  ■  .  ■-•-.■•.■ .  i 

alternatives  including  changes  in  (1)  objective,  (2) 
constraint,  (3)  criteria,  (4)  task  criticality  variables, 
(5)  psychological  variables,  (6)  task  criticality  variable 
weights,  and/or  (7)  utility  weights. 

After  users  have  explored  these  alternatives,  the  task-media 
assignments  can  be  finalized  and  entered  into  the  SDT. 


Appendix  B:  Overview  of  Automated  Resource  and  Cost 

Estimation  Technique 


This  appendix  provides  detailed  instructions  on  how  to  use 
the  Automated  Resource  and  Cost  Estimation  Technique 
(RCET).  A  more  detailed  description  of  RCET  is  contained  in 
the  User's  Guide:  Resource  and  Cost  Estimation  Technique. 
The  purpose  of  RCET  is  to  provide  Army  Training  analysts 
with  an  automated  tool  for  estimating  instructor 
requirements  and  institutional  training  costs  during  the 
earliest  phases  of  the  acquisition  process. 

The  Automated  Resource  and  Cost  Estimation  Technique  (RCET) 
is  designed  to  use  input  data  from  the  ETES  data  base 
management  system,  the  System  Description  Technology  (SDT). 
Actual  calculation  of  instructors  and  institutional  training 
course  cost  in  RCET  is  accomplished  by  using  the  VISICALC 
automated  worksheet  software  developed  by  Visicorp.  The 
VISICALC  worksheet  is  also  used  to  conduct  sensitivity 
analyses  of  key  parameters, 

B.l  CONCEPTUAL  OVERVIEW 

The  Resource  and  Cost  Estimation  Technique  has  three 
components : 

(1)  SDT  Interface  Software  -  this  software  is  used  to  select 
and  remove  data  from  the  SDT  and  to  format  the  data  for  use 
in  the  VISICALC  program.  In  addition,  it  is  used  to  copy 
the  results  of  the  VISICALC  program  back  into  the  SDT. 


(2)  Tailored  VISICALC  Worksheet  -  this  worksheet  contains 
the  equations  for  determining  number  of  instructors  and 
course  costs.  In  addition,  it  contains  all  of  the  commands 
needed  to  load  and  unload  the  SDT  input  file,  and  to  conduct 
sensitivity  analyses.  This  tailored  worksheet  saves  the 
user  from  the  somewhat  tedious  process  of  setting  up  a 
VISICALC  worksheet  and  command  structure. 

(3)  Manual  Procedures  -  these  procedures  describe  how  to 
develop  input  data  and  how  to  use  the  SDT  interface  software 
and  the  tailored  VISICALC  worksheet. 

There  are  two  major  products  of  RCET:  (1)  a  listing  of  the 
number  of  instructors  required  in  the  course  and  (2)  a 
listing  of  projected  costs  for  the  course.  An  example  of 
the  cost  elements  estimated  by  RCET  is  listed  in  Table  B- 
1.  These  are  the  same  cost  elements  used  in  the  Cost 
Analysis  Program  of  the  Army  TRADOC  Resource  Management 
(ATRM)  system. 

B.1.1  Calculation  of  Course  Costs' 

Costs  for  a  new  course  are  estimated  by  identifying  a 
canparable  existing  course,  obtaining  cost  data  from  this 
course  data  to  reflect  the  differences  in  key  resource 
requirements  (for  example,  number  of  students  and  number  of 
instructors)  between  the  comparable  course  and  the  new 
course. 

This  procedure  provides  estimates  of  course  costs  that  are 
(1)  empirically  based  and  (2'  suitable  for  the  types  of  high 
level  analyses  which  are  conducted  during  the  early  phases 
of  the  acquisition  process. 
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B.1.2  Calculation  of  Number  of  Instructors 

The  number  of  instructors  required  in  a  course  is  calculated 
by  an  automated  version  of  the  algorithm  listed  in  the 
Staff inq  Guide  for  U.S.  Army  Services  Schools  (DA  PAM  570- 


B.2  PROCEDURAL  OVERVIEW 

An  overview  of  the  procedures  in  the  Resource  and  Cost 
Estimation  Technique  is  provided  in  Figure  B-l. 

The  first  step  in  the  application  of  RCET  is  the 
identification  of  the  "reference"  course  or  the  comparable 
existing  cousre,  which  most  closely  resembles  the  task  and 
target  population  requirements  of  the  new  course. 

Procedures  for  identifying  a  reference  course  are  contained 
in  the  ETES  User  Guide  Manual  Procedures.  Once  the  reference 
course  has  been  identified,  cost  data  for  this  course  is 

obtained  from  the  Cost  Analysis  Program  ( MOS  Training  Cost) 
and  entered  into  the  SDT. 

Reference  course  information  is  also  used  in  the 

construction  of  the  quasi-program  of  instruction  (QPOI)  for 
the  new  course.  Included  in  the  QPOI  is  a  description  of 
the  methods  to  be  used  in  each  module  in  the  course,  and  the 
student- ins  true  tor  ratio  and  instructor  contact  hours 
associated  with  each  method.  Procedures  for  constructing  a 
QPOI  are  contained  in  the  ETES  User's  Guide.  This  same 

guide  contains  procedures  for  determining  the  number  of 
students  to  be  trained.  This  value  is  a  critical  factor  in 
the  determination  of  course  costs. 


Once  the  QPOI  has  been  instructed  for  the  new  course, 
information  from  the  QPOI  on  instructional  methods,  student/ 
instructor  ratios,  and  contact  hours  must  be  entered  into 
the  SDT.  When  this  is  completed  the  the  user  must  enter  the 
SDT,  enter  the  Applications  mode,  select  the  Resource  and 
Cost  Estimation  Techniques  (RCET ) ,  and  copy  the  reference 
cost  data  and  new  course  QPOI  data  onto  files  which  can  be 
read  into  the  VTSICALC  program. 

Once  the  VISICALC  input  files  have  been  developed,  the  user 
must  remove  the  SDT  software  diskette,  put  in  the  VISICALC 
software  diskettes  and  enter  the  VISICALC  program.  Once  into 
the  VISICALC  software,  a  few  simple  commands  may  be  used  to 
load  the  SDT  input  files  and  RCET  worksheet  into  the 
VISICALC.  When  this  is  completed,  a  small  selected  set  of 
resource  data  for  the  reference  and  new  course  must  be 
entered  into  the  SDT.  Course  costs  and  instructor 

requirements  may  then  be  calculated  by  executing  a  simple 
command  built  into  the  RCET  worksheet. 

After  examining  the  initial  estimates  of  course  costs  and 
instructor  requirements,  the  user  can  then  use  a  few 
commands  built  into  the  RCET  worksheet  to  conduct 
sensitivity  analyses  of  key  parameters.  When  these  analyses 
are  complete,  the  final  set  of  costs  for  the  new  course  can 
be  copied  onto  a  VISICALC  output  file.  The  user  can  then 
exit  the  VISICALC  software,  enter  the  SDT  software  and  copy 
the  output  file  into  the  SDT  data  base. 
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Appendix  C  -  Overview  of  Automated  Training  Planning 
and  Scheduling  Technique 


This  appendix  provides  an  overview  of  the  Automated 
Scheduling  and  Planning  Techniques  (APST).  A  more  detailed 
description  of  APST  is  provided  in  the  User's  Guide: 
Automated  Planning  and  Scheduling  Technique.  APST  is 
designed  to  assist  training  developers  in  describing  and 
monitoring  the  training  development  schedule  for  developing 
Army  weapon  systems.  APST  is  designed  to  be  used  with  the 
Visischedule  software,  which  is  an  automated  program  for 
describing,  monitoring  and  reporting  schedule  information 
and  for  conducting  critical  path  analyses  of  schedule 
events.  A  data  input  diskette,  describing  the  events 

required  in  the  Army's  Individual  and  Collective  Training 
Plan  ( ICTP ) ,  is  included  in  these  techniques.  This  data 
input  diskette  contains  detailed  information  on  the 
sequential  relationships  among  the  events  in  the  OICTP. 

APST  is  designed  to  make  it  relatively  easy  for  training 
developers  to  track  and  monitor  the  complex  relationships 
among  the  events  in  the  training  development  schedule.  In 
addition,  by  providing  an  automated  capability  to  monitor 
the  training  schedule,  it  should  aid  training  developers  in 
responding  quickly  and  efficiently  to  the  frequent  schedule 
changes  which  occur  during  the  develpoment  of  Army  weapon 
systems . 


C.l  CONCEPTUAL  OVERVIEW 


Construction  of  training  development  schedules  for  emerging 
systems  is  a  difficult  task.  Over  100  developmental  events 
are  listed  in  TRAD  PC  Reg  3  51-9.  The  sequential 

relationships  among  these  events  are  complex  and  are  not 
described  in  any  systematic  and  integrated  manner  in  TRADOC 
Reg  3  51-9. 

Further,  the  training  scheduling  process,  particularly 
during  the  early  phases  of  system  development,  is 
characterized  by  frequent  changes  and  updates. 
Determination  of  the  impact  of  these  changes  is  a  tedious 
and  time  consuming  process. 

APST  contains  techniques  for  using  automated  VisiSchedule 
software  to  track  and  monitor  training  develpoment  schedule. 
By  using  VisiSchedule,  the  training  developer  can  quickly 
and  efficiently  respond  to  changes  in  the  training 

development  scheudle.  Use  of  the  VisiSchedule  program  is 
facilitated  by  the  inclusion  of  an  input  data  diskette  which 
(a)  describes  the  events  in  the  training  development  process 
(as  specified  in  TRADOC  Reg  351-9),  (2)  describes  the 

temporal/ sequential  relationships  among  these  events,  and 
(3)  lists  the  expected  duration  of  these  events  for  a 
"typical"  major  Army  weapon  system.  This  data  diskette 
significantly  reduces  data  input  requirements  In  addition, 
it  eliminates  the  need  for  an  analysis  of  the  canplex 
sequential  relationships  among  training  development  events 
which  are  either  implicitly  or  explicitly  specified  in 
TRADOC  3  51-9. 
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Capabilities  of  VisiSchedule  Software 


As  applied  to,  the  training  development  process,  the 
VisiSchedule  software  can  be  viewed  as  providing  the 
following  capabilities: 

(1)  Allows  users  to  systematically  describe  an 
integrated  training  development  schedule  including 
information  on  training  development  events,  the 
sequential  relationships  among  these  events,  the 
duration  of  these  events,  the  manpower  (by  labor 
category)  required  to  accomplish  each  event,  and 
the  costs  (that  is,  salaries)  of  this  manpower. 

(2)  Allows  users  to  quickly  determine  the  impact  of 

changes  to  any  of  the  above  information. 

(3)  Allows  users  to  identify  the  "critical  path"  in 

the  training  development  schedule.  A  "critical" 

event  is  one  whose  delay  would  impact  completion 

of  the  whole  project. 

(4)  Allows  user  to  aggregate  events  to  determine  total 
manpower  requirements  (by  paygrade  or  occupational 
specialty)  and  to  determine  total  training 
development  costs. 

C.2  PROCEDURAL  OVERVIEW 


An  overview  of  the  procedures  for  using  the  automated 
planning  and  scheduling  techniques  is  provided  in  Figure  C- 


The  first  procedure  involves  setting  up  the  hardware  and 
software  needed  to  run  the  automated  techniques.  As  part  of 
this  procedure,  the  VisiSchedule  software  and  the 
accompanying  input  data  diskette,  describing  the  ICTP  events 
and  their  interrelationships  are  entered  into  the  computer. 

In  the  next  three  procedures  (2.1  and  2.3),  the  data  on  the 
input  data  diskette  describing  the  ICTP  events,  their 
interrelationships,  and  their  durations  is  reviewed  by  the 
user.  If  the  user  feels  that  the  existing  data  is 
acceptable  and,  thus,  is  an  accurate  description  of  his/her 
training  development  schedule,  the  user  can  proceed  directly 
to  the  output  report  procedure  (4.0).  However,  it  is  more 
likely  that  the  user  will  want  to  change  the  durations  or 
completion  deadlines  of  some  of  the  events;  the  events 
themselves  and  their  relationships  are  less  likely  to 
require  change. 

If  changes  are  required,  these  changes  may  be  made  using  the 
methods  described  in  procedures  3.1  to  3.4.  Procedure  3.4 
allows  users  to  enter  and/or  modify  data  on  manpower 
requirement  and  costs.  Information  on  manpower  requirements 
must  be  entered  by  the  user  since  this  information  is  too 
system  specific  to  include  in  the  input  data  diskette. 

In  procedure  4.0,  the  user  may  select  from  one  of  four 
different  output  reports  to  describe  the  training 
development  schedule.  After  examining  these  outputs,  the 
user  may  wish  to  conduct  tradeoff  analyses  or  sensitivity 
analyses  of  the  schedule  input  variables.  This  can  be 
accomplished  by  changing  the  input  parameters  through 
procedures  2.0  and  3.0. 
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This  appendix  provides  descriptions  of  the  six  ARI  projects 
which  are  directly  related  to  ETES  (1)  the  Hardware 
Procurement-Military  Manpower  (HARDMAN)  Methodology,  (2)  the 
Man  Integrated  System  Technology  (MIST),  (3)  the  Training 
Efficiency  Estimation  Model  (TEEM),  (4)  the  Training 
Developer's  Decision  Aid  ( TDDA )  ,  (5)  the  Model  for  the 
prediction  of  the  effectiveness  of  Training  Devices 
(TRAINVICE),  and  (6)  the  Army  Manpower  and  Personnel 
Requirements  Process  (ARMPREP).  More  details  on  these  six 
projects  are  provided  in  the  subsections  which  follow. 

D.l  MILITARY  MANPOWER  VERSUS  HARDWARE  PROCUREMENT  (HARDMAN) 

The  HARDMAN  methodologies  provide  the  Navy  and  the  Army  with 
a  systematic  means  of  estimating  human  resource  requirements 
(manpower,  personnel,  and  training)  during  the  earliest 
phases  of  the  Weapon  System  Acquisition  Process  (WSAP).  The 
integrated  models  and  data  bases  of  the  methodologies  are 
designed  to  (1)  assess  manpower,  personnel,  and  training 
requirements  for  proposed  sytems;  (2)  determine  the  impacts 
of  manpower,  personnel,  and  training  requirements;  and  (3) 
identify  and  evalute  tradeoffs  which  would  alleviate 
unfavorable  impacts.  DRC  has  also  extended  the  capability 
of  HARDMAN  through  increased  automation  and  the  development 
of  proprietary  software  packages.  These  advances  have 
significantly  cut  cost  and  response  time  and  more  closely 
aligned  the  output  products  of  HARDMAN  with  the  information 
needs  of  Program  Managers,  resource  sponsors  and  their 
staffs. 
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In  its  present  form,  DRC 1 s  HARDMAN  Methodology  is  composed 
of  six  interrelated  steps  which  can  oe  iterated  in  a  timely 
fashion  for  the  analysis  of  alternative  design  proposals. 
The  first  step  is  conerned  with  various  aspects  of  data 
collection,  generation,  formatting,  and  analysis.  It  also 
includes  the  establishment  of  an  automated  audit  trail  or 
program  record  of  all  input  data,  analyses  and  output 
products.  The  final  five  steps  involve  data  evaluation  with 
respect  to  acquisition  program  goals  and  constraints. 

The  establishment  of  the  consolidated  data  base  triggers  the 
methodology  and  ensures  that  all  analyses  pertaining  to  a 
new  weapon  system  use  a  common  data  source.  Consequently, 
consistent  definitions  and  collection  procedures  are 
employed  in  assembling  data  on  operator  and  maintainer 
functions,  the  weapon's  operational  .assion/scenario  and 
maintenance  concept,  its  cost  factors,  and  its  manpower  and 
training  sytem  support  requirements.  The  proposed  weapon 
system's  demand,  in  terms  of  manpower,  personnel,  and 
training  is  determined  in  Steps  2  through  4  and  compared  to 
the  projected  supply  of  such  resources  at  system  Initial 
Operational  Capability  (IOC)  in  Step  5.  Significant 
shortfalls  and  equipment  sources  of  high  resource  demand  are 
identified.  In  Step  6,  the  methodology  is  iterated  to 
examine  alternative  designs,  training  methods  and  media,  and 
total  force  tradeoffs.  Thus  the  program  manager  can 
participate  in  the  design  process  with  a  heightened 
awareness  of  the  training  and  manpower  demands  of  a  new 
weapon  system,  thereby  ensuring  its  supportabi li ty  as  well 
as  its  mission  capability. 

DRC's  HARDMAN  Methodology  has  been  used  on  a  wide  range  of 
all  three  Military  Services,  including  the  Shipboard 
Intermediate  Range  Combat  (SIRC.S);  the  1SD-41,  a  new  ci  tss 
of  guided  missile  destroyer  (DDG-51);  the  Submarine  Advanced 
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Combat  System  (SUBACS);  the  Army's  Divison  Support  Weapon 
System  ( DSW'S )  ;  the  Corps  Support  Weapon  System  (CSWS);  and 
the  Remotely  Piloted  Vehicle  (RPV);  and  in  modified  form 
with  the  Air  Force's  Combat  Identification  System  (CIS). 

These  applications  have  fully  exercised  all  of  the  analytic 
capabilities  of  the  methodology.  Additionally,  they  have 
demonstrated  its  capability  to  provide  timely,  accurate 
support  for  program  management  and  review.  Speci f ical ly , 
significant  or  problematic  human  resource  demand  generated 
by  equipment  design  has  been  targeted  early  in  program 
development;  alternative  system  and  subsystem  configurations 
have  been  evaluated  through  tradeoff  studies;  and  the 
methodology's  consolidated  data  base,  supported  by  carefully 
documented  resource  design  analyses,  has  been  used  to  build 
an  audit  trail  of  manpower,  personnel,  and  training  assess¬ 
ments  and  tradeoffs. 

D . 2  MAN  INTEGRATED  SYSTEMS  TECHNOLOGY  (MIST) 

The  United  States  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences  (ARI)  is  developing  a  technology  which 
integrates  manpower,  personnel  and  training  (MPT)  consider¬ 
ations  throughout  the  Weapon  System  Acquisition  Process 
(WSAP).  Its  goal  is  to  ensure  the  effective  planning  for 
and  utilization  of  projected  human  resources  for  operational 
readiness.  The  technical  activity  required  to  achieve  this 
objective  has  been  organized  into  a  major  research  and 
development  program  entitled  Man  Integrated  Systems 
Technology  (MIST).  DRC  has  been  selected  as  the  contractor 
for  MIST  which  is  an  extensive  (5  year,  32  man-years)  multi¬ 
million  dollar  effort. 

MIST  will  consist  of  the  necessary  technology  and  management 
procedures  to  address  the  following  considerations:  (1)  the 


treatment  of  human  resources  as  a  performance  and  cost 
factor  during  system  concept  formulation,  (2)  the  planning 
and  forecasting  of  manpower  information,  (3)  the  parallel 
development  of  associated  training  systems  with  weapon 
systems,  and  finally  (4)  the  specification  of  test  and 
evaluation  issues  to  ensure  human  resource  accountability, 
and  to  support  the  ASARC/DSARC  review  at  each  of  the  major 
development  milestones.  MIST  in  particular  will  integrate 
and  demonstrate  the  technical  relationships  among  these 
considerations,  and  provide  the  necessary  methodology  to 
ensure  their  effective  treatment  during  the  design  process 
from  a  performance,  manpower,  training  and  cost  point  of 
view.  In  addition,  MIST  will  demonstrate  how  these 

technical  considerations  are  related  and  responsive  to  the 
Army's  Life  Cycle  System  Management  Model  ( LCSMM ) ,  and  to 
relevant  Army  regulations  and  responsible  agencies.  MIST 
will  further  provide  a  comprehensive  framework  for  the 
assessment  and  integration  of  other  relative  technologies 
and  information  as  they  are  identified  or  developed. 

When  fully  developed,  MIST  will  serve  as  an  integrating 
mechanism  providing  efficient  interfaces  for  information 
exchange  and  feedback  among  concurrent  activities  and/or 
processes  involving  industry  and  government,  system  design 
engineers  and  support  planners.  Army  acquisition  managers, 
and  the  Army's  oversight  bureaucracies  in  the  Department  of 
Defense  and  the  Congress.  Hence,  MIST  will  provide  the 
design  and  implementation  of  more  comprehensive  and 
responsive  data  bases  and  technical/management  procedures 
employed  in  prime  and  support  system  development, 
acquisition  and  operation. 
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When  completed,  MIST  will  be  composed  of  three  principal 
components: 

o  Analytical  Tool  Designed  to  Support  Program 
Managers,  TRADOC  System  Managers,  Acquisition 
Managers  (e.g..  Weapon  System  Manager  (WSM)  in 
DARCOM ) ,  and  their  staffs/supporting  agencies; 


o  A  system-specific  data  base  to  provide  a  single 
point  source  for  all  man-machine  data  (input  to 
the  tools,  output  from  them  and  decision  support 
information  for  program  documentation  and  reviews) 
collected  for  an  emerging  system. 

o  A  management  information  system  to  link  MIST  users 
and  the  data  base  in  an  effective  communication 
network  within  the  framework  of  the  Life  Cycle 
System  Management  Model. 

Hence,  MIST  will  provide  an  important  new  capability  within 
the  Army.  This  capability  is  embodied  in  a  cost-and  time- 
efficient  system  for  human  resource  requirements  analysis 
early  in  weapon  system  development.  MIST  will  capture  the 
most  recent  advances  in  both  man-machine  technology  and 
automated  data  processing  to  provide  a  system  which  is  user- 
friendly  at  the  operational  level  and  responsive  to  the 
analysis  and  information  requirements  of  users  at  several 
levels  within  the  chain  of  command.  Finally  MIST  will  "fit 
into  the  real  world"  by  providing  effective  interfaces  to 
existing  management  information  systems  and  data  bases  for 
logistic  support  analysis,  training  system  development  and 
personnel  planning  in  DARCOM,  TRADOC  and  the  Army  Staff. 


D . 3  TRAINING  EFFICIENCY  ESTIMATION  MODEL  (TEEM) 


TEEM  is  an  interactive  computer-based  aid  designed  to  assist 
training  developers  in  early  evaluation  and  generation  of 
training  program  components.  At  the  foundation  of  TEEM  is  a 
set  of  psychological  variables  which  reflect  the  training 
information  contained  in  a  system  task  description.  This  is 
in  contrast  to  the  normal  textual  form  in  which  task 
descriptions  are  usually  developed.  A  representation  by 
variables  was  chosen  bee  use  early  task  descriptions  are 
often  not  in  standard  formats,  may  change  quickly,  and  may 
require  elaboration  by  design  engineers  or  other  subject 
matter  experts.  A  predetermined  set  of  standard  psycho¬ 
logical  variables  provided  a  starting  point  which  could  be 
depended  upon  for  input  into  the  fixed  operational  code  of  a 
computer  program. 

Initial  task  groups  corresponding  to  procedures  that  should 
be  trained  together  are  generated  through  an  algorithm  that 
clusters  tasks  based  upon  internal  similarity  along  the 
psychological  variable  dimensions.  After  clustering  is 
complete,  a  task  group  has  maximum  difference  between  task 
clusters.  The  clusters  serve  as  an  automated  estimate  for 
use  during  the  initial  formulation  of  a  training  program. 
Task  groups  are  then  modified,  if  needed,  based  on  the 
expertise  of  the  analyst  or  outside  mitigating  factors  such 
as  management  and  cost  constraints  that  would  normally  not 
be  included  in  the  psychological  variables. 

Selection  of  concrete,  costable  components  such  as  media  and 
method  of  training  are  based  upon  the  similarity  of  a  group 
of  task  variable  requirements  to  the  ability  of  a  media  or 
method  to  manipulate  psychological  variables  during 
training.  Selection  is  accomplished  by  representing  each 
training  program  component  by  a  description  in  the  same  set 
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of  psychological  variables.  A  task  is  also  coded  as  a 
vector  of  variables  with  an  assocated  variable  value  for 
each  dimension  representing  applicability  or  non¬ 
applicability.  Selection  of  a  program  component  is 
performed  by  automatically  matching  each  task  to  each 
potential  component  and  selecting  the  component  with  the 
higher  match  of  critical  variables. 

If  an  already  existing  training  component  is  being 
evaluated,  the  matching  process  is  utilized  in  reverse  to 
generate  a  figure  of  aggreement  between  the  theoretical 
maximum  possible  under  the  task  and  hardware  conditions  and 
the  actual  match  under  a  particular  alternative.  The 
normalized  numeric  value  of  this  match  is  called  the 
efficiency  ratio. 

After  efficiency  ratios  have  been  generated  for  potential 
training  programs  a  final  selection  for  the  best  training 
approach  is  made  based  on  pair  values  of  gross  dollar  costs 
and  their  efficiency  ratios.  Together  the  values  represent 
an  early  form  of  cost-effectiveness  ratio  for  each  approach. 
The  best  candidate  programs  are  selected  for  detailed  cost 
analysis  through  a  much  more  complex  accounting  program  and 
given  increased  program  specification. 

TEEM  was  constructed  with  the  realities  of  the  early 
training  situation  in  mind.  It  does  not  assume  extensive 
and  accurate  information.  It  is  "user  friendly"  with  step- 
by-step  guidance  to  individuals  unfamiliar  with  micro¬ 
computers.  It  is  designed  to  accommodate  local  school  needs 
and  training  development  procedures  without  major  program 
changes.  It  is  also  flexible  enough  to  be  quickly  modified 
based  upon  improvements  in  psychological  research  and  field 
experience . 
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D . 4  TRAINING  DEVELOPER'S  DECISION  AID  ( TDDA ) 


The  Training  Developer's  Decision  Aid  (TDDA)  was  designed  to 
assist  training  development  specialists  in  applying  the 
Instructional  Systems  Design  (ISD)  process.  The  ISD  process 
is  a  comprehensive  technique  for  the  systems  approach  to 
development  training.  Applied  Science  Associates,  Inc. 
(ASA),  working  in  collaboration  with  the  Army  Research 
Institute  Field  Unit  at  Fort  Bliss,  Texas,  has  developed, 
tested,  revised,  computerized,  and  conducted  tryouts  of  the 
model  at  five  Army  schools. 

The  TDDA  is  intended  to  be  an  aid  to  training  developers  who 
need  technical  assistance  in  designing  or  redesigning 
training  and  to  streamline  the  process  of  making  training 
decisions.  The  TDDA  is  procedural  in  nature;  it  leads 
training  developers  through  a  series  of  procedures  which 
specify:  (1)  what  decisions  are  to  be  made,  (2)  a  rational 

order  for  decisions,  and  (3)  the  nature  and  sources  of 
information  required  to  make  valid  training  decisions.  The 
model  also  serves  to  organize  and  retain  information  that  is 
gathered  and  processed  during  the  course  development 
process . 

The  TDDS  model  can  be  used  with  partial  or  complete  task 
lists  or  performance  objectives.  It  is  not  necessary  to 
redo  the  job  analysis  in  order  to  use  the  TDDA  process. 
This  reflects  a  training  development  philosophy  that  a 
thorough  analysis  of  the  job  early  in  the  training 
development  process  is  essential  for  making  valid  training 
development  decisions. 


★ 


Derived  from  Frede r ickson ,  et  al  (1981). 


The  Job  Analysis  Module  requires  the  greatest  input  form 
subject  matter  experts  (SMEs).  Those  who  hold,  the  MOS  being 
analyzed  and  selected  to  serve  as  SMEs  are  expected  to  be 
expert  job  performers  with  at  least  eighteen  months  of 
recent  on-the-job  experience  at  an  operational  site.  This 
is  an  essential  requirement  for  the  successful  application 
of  the  model.  The  model  is  sensitive  to  quality  of  the 
input  data  and  information,  since  all  three  major  decision 
(what,  where  and  how  to  train)  are  directly  tied  to  input. 

There  are  two  separate  parts  to  the  SME  input  in  the  Job 
Analysis  Module.  First,  the  initial  task  list  is  generated 
by  exercising  function  analyzers.  Then,  the  task  list  is 
verified  as  correct  and  complete.  This  should  be  carried 
out  by  a  second  set  of  SMEs.  At  the  time  the  tasks  are 
verified,  the  remainder  of  the  input  information  is  obtained 
from  the  SMEs.  The  output  of  this  module  provides  the 
answer  to  the  question,  "What  should  be  trained  to  prepare 
someone  to  perform  in  their  MOS?" 

The  Functional'  Learning  Requirements  Module  requires  inputs 
from  two  sources,  MOS  SMEs  and  course  development  instructor 
personnel  in  order  to  make  decisions  of  how  to  train.  The 
information  generated  in  this  module  is  used  by  the  course 
developer  to  specify  the  functional  characteristics  of  the 
training  program.  The  emphases  in  specifying  these 
characteristics  are  on  how  best  to  communicate  the  learning 
objectives,  job  context  and  consequences  of  task 
performance,  and  on  providing  for  efficient  practice  for 
acquiring  special  job  skills. 


Development  of  the  course  requires  that  the  relationships 
between  tasks  be  determined.  The  last  SME  input  provides 
the  information  for  describing  these  relationships.  Tasks 
are  related  in  three  ways.  First,  the  same  skills  may  be 
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required  in  the  performance  of  different  tasks.  Knowing 
which  skills  are  redundant  allows  for  more  efficient 
scheduling  and  sequencing  of  instructional  elements. 
Second,  some  tasks  must  be  performed  on  the  job  before  other 
tasks  can  be  performed.  For  example,  a  piece  of  equipment 
cannot  be  repaired  before  the  cause  of  the  malfunction  is 
isolated.  Or,  an  intelligence  report  cannot  be  prepared 
before  the  intelligence  information  and  data  are  analyzed. 
The  output  of  these  tasks  are  inputs  to  the  subsequent 
tasks.  The  third  relationship  is  merely  chronological.  One 
task  must  be  performed  prior  co  a  second  one  but  there  is 
not  direct  output-input  dependency.  For  example,  a  weapon 
system  has  to  be  emplaced  (a  collective  task)  and  made  ready 
for  action,  before  it  can  be  used  to  engage  hostile  targets, 
which  may  or  may  not  occur. 

The  dependency  relationship  information  is  provided  by  SMEs 
and  is  then  used  to  build  a  task  dependency  hierarchy.  The 
product  is  considered  to  be  a  general  course  structure  upon 
which  the  course  map  can  be  built.  The  task  dependency 
information  is  generated  in  the  Functional  Learning 
Requirements  Module,  but  is  analyzed  in  the  Structure 
Designation  Module. 

Information  and  data  from  several  earlier  inputs  are  used  in 
the  last  module  to  make  decisions  as  to  where  the  training 
should  be  conducted.  The  task  criticality  data  is  one  of 
the  primary  inputs  to  this  decision.  Additionally,  the 
instructional  setting  characteristics  and  school  context 
information  are  used  in  reaching  these  conclusions. 

The  TDDS  model  then  is  used  to  make  three  major  decisions: 
what  to  train,  how  to  train,  and  where  to  train.  Some 
decisions  are  made  algorithmically  and  others  are 
heuristically  determined. 
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D .  5  A  MODEL  FOR  THE  PREDICTION  OF  THE  EFFECTIVENESS  OF 
TRAINING  DEVICES* 

TRAINVICE  is  a  methodology  for  the  systematic  assessment  of 
the  characteristics  of  training  devices  under  development. 
The  model  is  based  on  the  assumption  that  certain  attributes 
to  be  assessed  in  the  training  situation  will  lead  to 
transfer  of  training  to  the  operational  situation.  There¬ 
fore,  the  higher  the  rating  on  the  assessment  factors,  the 
higher  the  transfer  that  will  take  place  and  the  more 
effective  the  device.  The  model  provides  a  framework  for 
the  making  of  these  judgments.  The  three  variables  entering 
into  the  assessment  are:  (1)  the  transfer  potential  of  the 
device,  (2)  the  learning  deficit  to  be  overcome  and  (3) 
instructional  effectiveness.  As  with  any  model,  its 
effectiveness  depends  on  the  adequacy  of  the  input  data. 
Inputs  into  the  model  consists  of  descriptions  of  tasks  and 
subtasks  represented  in  the  operational  situation,  as 
circumscribed  by  the  training  objective,  and  those 
represented  in  the  training  device.  The  controls  and 
displays  and  their  functions  for  both  situations  are  listed. 
In  addition,  the  skills  and  knowledges  involved  in  each 
subtask  in  the  operational  situation  are  formulated  for  use 
in  the  model.  Using  these  inputs,  judgments  are  made  using 
rating  scales.  The  subtasks  in  the  two  situations, 
operational  and  training,  are  compared  to  ascertain  if 
provision  is  made  for  representation  of  the  subtasks  in  the 
training  device  in  the  commonality  analysis.  Next,  the 
displays  and  the  controls  for  both  situations  are  compared 
on  physical  and  functional  similarity.  The  more  similar  the 
display  or  control  in  the  training  device  is  to  the 
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operational  situation,  the  higher  the  score.  This  is  based 
on  the  premise  that  the  greater  the  physical. or  functional 
similarity,  the  greater  the  transfer  of  training  that  will 
result.  Physical  similarity  refers  to  the  appearance  and 
physical  aspects  of  the  displays  and  controls  involved; 
i.e.,  their  "fidelity";  functional  similarity  involved  in 
the  operation  of  control,  in  information  processing  terms. 
The  learning  deficit  analysis  is  based  upon  (1)  the  assess¬ 
ment  of  the  level  of  proficiency  in  each  skill  or  knowledge 
for  the  students  upon  entering  the  training  situation,  (2) 
the  desired  level  of  proficiency  in  each  skill  or  knowledge 
for  the  students  upon  leaving  the  training  situation,  and 
(3)  the  difficulty  (in  terms  of  training  time)  of  training 
in  the  skills  or  knowledges  involved  in  a  subtask.  This 
analysis  yields  a  weighted  learning  deficit  for  each 
subtask.  The  judgments  concerning  the  level  of  each  skill 
or  knowledge  are  made  using  scales  adapted  from  Demaree 
(1961).  The  last  analysis  involved  in  the  TRAINVICE  model 
is  an  assessment  of  how  well  the  training  device  adhers  to 
"good"  training  techniques.  In  order  to  perform  this 
analysis,  each  of  the  subtasks  is  cast  into  one  or  more 
categories  of  behavior.  These  categories  are  those  of 
Braby,  et  al  (1972),  which  are  derived  for  an  earlier 
behavioral  categorization  by  Willis  and  Peterson  (1961). 
For  each  of  the  behavioral  categories  represented  in  the 
subtask,  a  list  of  guidelines,  also  those  of  Braby,  et  al. 
(1972),  are  consulted  and  judgments  made  of  the  degree  to 
which  the  guidelines  are  followed,  or  not  followed,  relative 
to  the  manner  in  which  the  subtask  is  represented  in  the 
training  device.  The  guidelines  are  broken  up  into  those 
dealing  with  the  stimulus,  response,  and  feedback  aspects  of 


the  training  situation. 
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preceding  rating,  are  then  fed  into  an  equation  to  formulate 
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an  index  of  prediction  of  training  effectiveness,  ranging 
from  0  to  1.  This  equation  is  as  follows: 


(Ci  x  Si  x  Ti  x  Di) 


Di 

where  C  is  task  commonality,  S  similarity,  T  training 
techniques,  and  D  the  training  deficit  scores  for  each 
subtask.  The  equation  was  derived  from  a  transfer  of 

training  equation  of  Gagne,  Foster  and  Crowley  (1943),  which 
was  for  use  with  empirical  data,  while  the  TRAINVICE  extra¬ 
polation  deals  with  judgments  made  concerning  aspects  of  a 
device  assumed  to  bring  about  subsequent  transfer  of 
training. 

A  validation  study  has  been  performed  on  the  model, 
utilizing  data  obtained  during  the  course  of  two  field 
studies  as  criteria  against  which  to  compare  the  predictions 
derived  from  use  of  the  model  (Wheaton,  et  al. ,  1976).  The 
devices  were  tank  gunnery  trainers  involved  with  burst-on- 
target  techniques  and  tracking  with  the  main  gun  of  the 
M60A1  tank.  In  each  case,  the.  prediction  of  no  differences 
between  the  training  devices  involved  was  found  to  be 
consistent  with  the  equivalence  in  transfer  actually  found 
in  utilization  of  the  various  devices.  This  was  felt  to  be 
a  promising  but  not  definitive  finding. 

In  order  to  obtain  additional  validation  data  on  the  model, 
and  also  to  obtain  experience  in  utilization  of  the  model  to 
determine  if  there  were  aspects  that  might  be  changed  in 
order  to  enhance  the  practicality  of  utilization  of  the 
model,  the  Army  Research  Institute  personnel  applied  the 
model  to  two  maintenance  trainers  undergoing  evaluation  at. 
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the  Army  Ordnance  Center  and  School .  This  afforded  the 
opportunity  to  obtain  data  within  a  different  context  than 
that  dealt  with  by  gunnery  trainers. 

These  trainers  were  concerned  with  automat ive  trouble¬ 
shooting.  No  difference  in  training  effectiveness  was 
predicted  for  the  two  trainers,  which  agreed  with  the 
results  of  the  empirical  evaluation.  Various  aspects  of  the 
model  which  caused  difficulty  in  its  utilization  were  noted 
and  influenced  the  development  of  the  modified  version  .  Tn 
addition,  ARI  conducted  a  three-day  workshop,  in  which  the 
developers  of  the  original  model  and  individuals  who  had 
utilized  the  model  or  had  an  interest  in  its  utilization 
participated,  and  this  furnished  further  ideas  for  possible 
mod i f ica t ion. 

D.  6  ARMY  MANPOWER  AND  PERSONNEL  REQUIREMENTS  PROCESS 
( ARMPREP) 

The  Systems  Manning  Technical  Area  of  the  Systems  Research 
Laboratory,  US  Army  Research  Institute,  is  promoting  a 
research  thrust  known  as  Manned  System  Integration.  One 
major  component  of  this  program  is  the  development  of 
improved  procedures  for  the  manpower  planning  process.  The 
major  objective  of  this  proposed  research  is  the  development 
of  new,  innovative  and  effective  techniques  to  support  the 
determination  of  manpower  and  personnel  requirements. 
ARMPREP  will  include:  methodologies  to  accurately  estimate 

manpower  quantity  and  skill  level  of  personnel  through 
derivation  of  behavioral  requirements  and  subsequent 
translation  into  Military  Occupational  Specialty  ( MOS )  and 
other  relationships;  techniques  to  aggregate  manpower  demand 
of  new  systems  for  comparison  with  available  supply; 
requirements  and  attributes  of  a  manpower  requirements 
management  information  system;  and  finally,  the  specifica- 
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tion  for  a  computer  interactive  system.  ARMPREP  is  intended 
to  provide  tools  for  potential  integration  into  the  current 
manpower  planning  process  by  cognizant  Army  agencies.  In 
essence,  ARMPREP  has  four  principal  components  each  with  its 
own  identifiable  research  products.  These  four  components 


o  Manpower  and  Personnel  Requirements  Determination 

Methodologies  (MANPERS). 

o  Aggregation  Procedures  of  Manpower  Demand  (TOTAL 

MANPERS ) . 

o  Requirements  for  a  Manpower  Requirements 

Management  Information  System  (MARMIS). 

o  Computer  Interactive  System  for  determination  of 

Manpower  arid  Personnel  Requirements  (AUTO 
MANPERS)  . 

MANPERS 

The  MANPERS  component  will  provide  the  techniques  and 
methodologies  to  formally  quantify  manpower  and  personnel 
requirements.  These  formal  methodologies  will  aid  in  the 
timely  and  accurate  determination  of  manpower  and  personnel 
requirements.  In  addition,  these  methodologies  are  intended 
to  standardize  the  manpower  and  personnel  requirements 
determination  process  to  provide  information  of  a  consistent 
nature  of  use  to  Army  personnel  subsystem  developers. 
Specific  aspects  of  the  MANPERS  component  will  consist  of 
the  following  attributes: 


(a)  The  development  of  a  taxonomic  model  to  derive 
behavioral  requirements  based  upon  new  system  Task 


Descriptive  Data  (TDD)  and  translation  of  these 
behavioral  requirements  into  MOS  and  associated 
decisions.  The  model  will  define  the  requisite 
data  to  be  included  .  This  input  data  will  be 
obtained  from  sources  such  as  LSA,  Task  and  Skill 
Analyses  (TASA),  system  requirements  and  doctrine 
documents,  engineering  design  data  and  training 
plans.  The  model  will  specify  the  leve)  of 
specificity  of  data  congruent  with  each  phase  of 
the  LCSMM .  In  addition,  the  model  will  identify 
the  process  by  which  the  behavioral  requirements 
are  derived  (i.e.,  the  taxonomy)  and  the  means  by 
which  the  level  of  data  specifity  is  embraced  by 
the  taxonomy.  Finally,  the  MANPERS  methodologies 
will  guide  the  selection  of  personnel  for  a  given 
job  in  a  new  system  based  upon  the  behavior  (or 
performance)  expected  of  him,  as  opposed  to  the 
current  method  which  involves  the  preselection  of 
MOS  category,  previous  to  and/or  without  rigorous 
documentation  to  support  such  a  choice. 

Integration  with  events  and  activities  of  the 
LCSMM,  ILS  (LSA)  and  ISD.  The  MANPERS  process 
will  provide  the  necessary  manpower  input  to 
decision  makers  at  critical  system  development 
milestones  to  encourage  optimization  of  manpower 
planning.  Where  possible  it  will  support  the 
determination  of  manpower  related  life  cycle  and 
support  costs  relative  to  specific  systems. 
Through  involvement  with  the  ISD  model,  the 
MANPERS  component  will  support  a  margin  of 
requirements  for  new  equipment  training  (of  the 
Material  Developer)  with  training  requirements  (of 
the  Combat  Developer)  as  identified  by  specific 
schools.  Finally,  the  MANPERS  component  will 


serve  to  provide  increased  accuracy  of  information 
used  in  development  of  draft  plan  TOEs  (as  well  as 
throughout  the  TOE  planning  process). 

(c)  A  final  aspect  of  the  MANPERS  component  is  the 
development  of  a  MANPERS  manual  to  include:  job 

aids,  examples,  procedural  models,  instrument 
formats  and  presentation  methods  for  thorough 
manpower  and  personnel  requirements  determination. 
The  MANPERS  manual  will  provide  a  tool  for  quick 
responses  to  needs  to  generate  quality  information 
on  a  timely  and  economical  basis.  The  MANPERS 
manual  will  facilitate  the  standardized  determin¬ 
ation  and  presentaiton  of  requirements  to  increase 
the  utilization  of  information  by  decision  makers. 
Finally,  the  MANPERS  manual  will  provide  a 
competent  training  tool  to  producers  of  manpower 
and  personnel  requirements  (especially  for  those 
new  to  the  job  and  content  area). 

TOTAL  MANPERS 

The  TOTAL  MANPERS  component  will  provide  the  techniques  for 
aggregation  of  manpower  demand  for  comparison  with  available 
supply  data.  Manpower  demand  is  defined  as  quantity  of 
manpower  by  quality  of  personnel  (e.g.,  skill  level)  both 
within  and  across  systems.  In  short,  it  is  a  quantity  by 
quality  of  manpower  comparison  between  demand  and  supply. 

Specific  aspects  of  the  TOTAL  MANPERS  component  include: 

(a)  Development  of  technqies  which  can  provide 
quantity  by  quality  loadings  of  manpower  demand 
for  a  specific  system.  The  data  used  by  TOTAL 
MANPERS  procedures  will  be  an  extension  of  the 


basic  MANPERS  tools  by  relating  some  paradigm  of 
personnel  performance  or  behavioral  requirements. 


(b)  Another  aspect  of  this  component  will  be  to 
provide  techniques  which  aggregate  information  for 
quantity  by  quality  of  manpower  across  systems 
through  comparison  of  manpower  demand  loadings 
held  in  common  with  personnel  performance  or 
behavioral  requirements. 

(c)  A  final  aspect  of  the  TOTAL  MANPERS  component  will 
be  its  coordination  with  events  and  activities  of 
the  LCSMM,  the  PPBS  (through  SACS  and  TOE 
planning)  and  I LS  to  insure  the  timely 
determination  of  requisite  manpower  information, 
and  authorizations,  as  well  as  changes  to  force 
structure  and  composition. 

MARMIS 

The  MARMIS  component  will  provide  the  manpower  management 
procedures  to  integrate  the  MANPERS  and  TOTAL  MANPERS 
components  with  the  Army  manpower  related  documentation 
process.  In  addition,  current  manpower  management  and 
documentation  procedural  deficiencies  and  recommended  fixes 
will  be  identified.  The  most  significant  activity  in  this 
component  is  to  support  current  Army  actions  in  management 
information  systems  developments  relating  to  the  manpower 
planning  process  through  the  specification  of  the  require¬ 
ments  and  attributes  of  MARMIS.  Specific  aspects  of  the 
MARMIS  component  include: 

(a)  Determination  of  manpower  management  and 

documentation  procedures  to  facilitate  the 

utilization  of  manpower  and  personnel  requirements 
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methodologies  and  aggregation  techniqes  so  as  to 
improve  the  timeliness  and  quality  of  requirements 
generated . 


(b)  Identification  of  means  to  improve  accountability 
of  manpower  and  personnel  requirements  documenta¬ 
tion  to  identify  potential  economies  in  management 
and  documentation  flows.  This  will  take  into 
account  organizational  and  technical  complexities 
which  serve  to  work  against  desired  results.  In 
addition,  these  efforts  will  facilitate  improved 
manpower  planning  as  well  as  to  focus  upon 
deficiencies  for  possible  resolution. 

(c)  Means  to  control  manpower  related  information  will 

be  developed  so  that  many  activities  concerned 
with  other  interests  (e.g.,  LCSMM,  SACS  and  TOE 
planning,  and  ILS)  can  be  fully  coordinated  with, 
such  that  appropriate  milestone  are  achieved  on  a 
timely  basis  and  the  requirements  of  other 

activities  are  met  as  they  relate  to  manpower 

concerns . 

AUTO  MANPERS 

The  AUTO  MANPERS  component  will  provide  the  basis  for 

development  of  a  computer  interactive  system  for  determining 
manpower  and  personnel  requirements  by  developing  the 

requirements,  attributes  and  specification  for  AUTO  MANPERS. 
AUTO  MANPERS  is  intended  to  provide  a  basis  for  MANPERS  an 
TOTAL  MANPERS  methodologies.  Specific  aspects  of  the  AUTO 
MANPERS  component  will  support  development  of  a  proposed 
computer  interactive  system  by  addressiny  the  following: 


(a)  Ease  of  documentation,  update  and  edit  of  require¬ 

ments  through'  use  of  an  online,  interactive  ADP 
system  which  provides:  continuous  availability  to 
authorized  users,  embedded  training  or  job  aids, 
and  quick  response  cross-referencing  capabilities 
with  related  manpower  documentation.  These 

capabilities  should  lead  to  increased  accuracy  and 
timeliness  of  requirements  production  and 
utilization  as  well  as  potential  economies  in 
administrative  support. 

(b)  Another  potential  aspect  of  this  component  is  the 
development  of  a  built-in  analytic  model  and 
mathematical  capability  to  increase  the  rigor  of 
manpower  requirements  determination.  For  example, 
whenever  mathematical  calculations  or  transforma¬ 
tions  are  required  the  formulas  should  be  readily 
accessed  on  the  system. 

It  should  be  noted  that  a  training  device  can  be  considered 
to  be  a  large  system  in  and  of  itself.  Thus,  it  has  its  own 
associated  resource  requirements.  Estimation  of  these 

additional  resource  requirements  is  beyond  the  current  scope 
of  ETES.  However,  it  is  expected  that  these  resource 
requirements  could  be  estimated  by  comparability  analysis. 
Input  to  the  determination  of  training  device  resource 
requirements  is  provided  by  the  task  requirements  associated 
with  each  device  which  are  determined  in  Procedure  2.0  and 
the  instructional  programs  developed  during  Procedure  4.5. 
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Appendix  F.  List  of  Potential  Performance  Measures. 


Speed 

Acceleration 

Range 

Vertical  Manueverability 

Horizontal  Manueverability 

Carrying  Capacity 

Rate  of  Fire 

Firing  Capacity 

Firing  Payload  Range 

Firing  Accuracy 

Communication  Range 

Communication  Rate 

Communication  Capacity 

Communication  Distortion/Accuracy 

Detection  Range 

Detection  Capacity 

Detection  Accuracy 

Recovery  Time 

Probability  of  Survival 

Memory  Capacity 

Processing  Speed 

Processing  Accuracy 

Processing  Capacity 

Operational  Availability 

Achieved  Availability 

Inherent  Availability 

Availability 

Mean  Time  Between  Failures  (MTBF) 

Summary  Measures  —  Elapsed  Maintenance  Time 
Mean  Corrective  Maintenance  Time 
Mean  Preventive  Maintenance  Time 
Mean  Active  Maintenance  Time 
Maintenance  Downtime 

Mean  Time  Between  Maintenance  Actions  (MTBMA) 

Mean  Active  Corrective  Maintenance  Time 
Mean  Active  Preventive  Maintenance  Time 
Maximum  Active  Corrective  Maintenance  Time 
Turn  Around  Time 
Summary  Measures  —  Manhours 

Mean  Maintenance  Manhours  per  Equipment  Operating  Unit 

Mean  Maintenance  Manhours  per  Maintenance  Action 

Mean  Maintenance  Manhours  per  Month  of  System  Operation 

Mean  Corrective  Maintenance  Manhours 

Mean  Preventive  Maintenance  Manhours 

Detailed  Measures  (per  Task  and  Task  Type) 

Elapsed  Time 
Maintenance  Hours 
Number  of  People-  Performing 
Frequency  of  Maintenance  Action 
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APPENDIX  G 


DESCRIPTION  OF  TRAINVICE 


This  appendix  provides  a  description  of  the  TRAINVICE 
procedures.  The  description  is  taken  f rom  Training  Device  Require- 
ments  Document  Guide. 


G-l 


Evaluate  Theoretical  Device  Concepts--If  no  existing  devices  approximate 
the  proposed  device's  requirements  or  if  alternative  existing  device  con¬ 
cepts  are  available,  one  may  wish  to  derive  new  alternative  concepts  that 
have  no  existing  hardware  representation.  Theoretical  device  concepts  are 
those  device  concepts  for  which  no  actual  training  device  equipment  now 
exists  but  for  which  such  equipment  could  theoretically  be  constructed.  As 
with  existing  device  concepts,  theoretical  device  concepts  are  specified  by: 


G-2 


•  Tasks 


•  Task  elements 

•  Controls  and  displays 

For  theoretical  device  concepts,  these  data  are  all  defined  in  the  task  train¬ 
ing  requirements  already  developed,  A  theoretical  device  concept  specified 
by  all  of  the  task  training  requirements  would  represent  the  "ideal"  device; 
by  definition,  it  would  meet  all  task  training  requirements.  Because  of 
practical  constraints  (e.g. ,  cost,  development  time),  it  may  not  be  possible 
to  develop  the  "ideal"  device.  Some  trade-offs  must  be  made.  These  affect 
tasks,  elements,  controls,  and  displays  which  specify  a  device.  Thus,  for 
practical  reasons,  it  is  desirable  to  be  able  to  evaluate  alternative  theor¬ 
etical  device  concepts.  The  procedures  described  below  allow  this;  regard¬ 
less  of  whether  existing  or  theoretical  alternative  device  concepts  are  being 
evaluated,  the  procedures  are  applied  in  the  same  way. 

Analyze  Alternative  Device  Effectiveness 

Task  Co  nmonality- -Table  27  is  used  to  analyze  each  task  requiring  training 
on  a  training  device.  The  table  is  used  in  deriving  a  task  commonality  index 
for  each  training  device  or  training  device  concept  under  consideration.  The 
following  procedure  is  applied  to  one  task  at  a  time. 

1.  List  the  task  elements  down  the  left  column  of  Table  27. 

2.  List  the  training  devices  or  training  device  concepts  to  be 
assessed  across  the  top  of  Table  27. 
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TABLE  27.  TASK  COMMONALITY  ANALYSIS 


NAME  OF  TASK 


3.  For  each  device  or  device  concept  under  consideration  list 
on  a  separate  piece  of  paper  the  elements  of  the  task  as  they 
would  be  trained  on  that  device. 

4.  For  each  task  element  in  the  left  column  of  Table  2  7 
determine  whether  a  device  enables  the  trainee  to  prac¬ 
tice  that  task  element  by  examining  the  separate  list  of 
task  elements  for  that  device. 

5.  If  the  training  device  does  allow  practice  of  that  operational 
task  element,  then  enter  a  "1"  in  the  appropriate  cell  for 
that  device. 

6.  If  the  particular  task  element  is  not  represented  in  the 
training  device,  either  because  the  task  is  truncated  or 
simplified,  then  enter  a  "0"  in  the  appropriate  cell. 

7.  Repeat  this  procedure  for  each  task  element  of  the  training 
task. 

8.  Repeat  the  entire  procedure  for  each  training  device  or 
device  concept  under  consideration. 

9.  For  each  training  device  or  device  concept  under  consid¬ 
eration,  total  the  number  of  "l"s  (i.e.,  the  number  of  task 
elements  for  which  the  training  device  allows  the  trainee 
to  actually  practice  the  training  task). 

10.  Enter  the  total  number  of  "l"s  for  each  device  in  Table  27. 

.11.  Derive  the  task  commonality  index  value  b>  dividing  the 
total  number  of  "l"s  by  the  total  number  of  task  elements 
in  the  training  task  plus  the  total  number  of  task  elements 
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unique  to  a  device  which  are  not  related  to  the  training 
* 

requirements.  The  number  of  elements  unique  to  a  device 
and  not  needed  is  included  in  the  formula  to  reduce  the  final 
effectiveness  score  for  those  devices  offering  greater  cap¬ 
ability  (and  therefore  cost  more)  than  actually  required. 

A  separate  task  commonality  table  is  used  for  each  task.  How  faithfully  the 
element  is  represented  in  a  training  device  is  considered  separately. 

Physical  Similarity — A  physical  similarity  analysis  is  based  on  the  physical 
similarity  or  fidelity  of  displays  (cues)  and  controls  (responses)  in  a  training 
device  relative  to  those  in  the  operational  equipment.  The  following  proce¬ 
dure  is  applied  to  each  task: 

1.  List  the  controls  and  displays  involved  in  task  performance 
down  the  left  column  of  Table  28. 

2.  List  the  training  devices  or  training  device  concepts  to  be 
assessed  across  the  top  of  Table  28. 

3.  For  each  control  or  display  listed  in  the  left  column  of 
Table  2  8,  rate  how  well  it  is  physically  represented  in  each 
training  device  or  device  concept  under  consideration.  Ease 
your  ratings  of  physical  similarity  on  the  four-point  scale 
given  in  Table  29. 


In  some  cases,  additional  elements  which  aid  training  may  be  present  (for 
example,  augmented  feedback).  The  present  use  of  "unique"  element  does 
not  include  these  training-related  extra  elements.  If  extra  elements  are 
present  which  enhance  training  of  the  required  task,  these  should  be  noted 
for  later  reference  in  overall  evaluation  of  the  device  concept. 
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TABLE  28.  PHYSICAL  SIMILARITY  ANALYSIS 


NAM  OF  TASK 


REQUIRED  TASK 

CON  ROLS  AND  DISPLAYS 

DEVICE  A 

DEVICE  B 

DEVICE  C 

1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 

9. 

10. 

Sum  of  Physical  Similarity  Ratings 

Physical  Similarity  Index 

Physical  Similarity 
Index 


3(  =  of  Ra'd  Com 


Sum  of  Similarity  Ratings  for  Device _ . 

A  Oisolavs)  -  of  Controls  &  Displays  Unique  to  Device ) 


TABLE  29.  PHYSICAL  SIMILARITY  RATING  SCALE 


Rating 

3 


2 

1 


Def ini tion 

Identi cal .  The  trainee  would  not  notice  a  difference  between 
the  training  device  control  or  display  and  the  operational 
control  or  display  when  he  moves  from  the  training  to  the  job 
situation.  Include  for  consideration  the  location,  appearance, 
feel,  and  any  other  physical  characteristics.  Ignore  the  amount 
and  quality  of  information  transmitted. 

Simi 1 ar.  There  would  be  a  small  noticeable  difference  for  the 
trainee  between  the  training  device  control  or  display  and  the 
operational  control  or  display,  but  he  would  be  able  to  perform' 
the  task.  There  might  be  a  decrement  in  performance,  but  any 
such  decrement  would  be  small  and  readily  overcome. 

Dissimilar.  There  would  be  a  large  noticeable  difference  quite 
apparent  to  the  trainee,  between  the  training  device  control  or 
display  and  the  operational  control  or  display  and  a  large  per¬ 
formance  decrement,  given  that  the  trainee  could  perform  at  all. 
Specific  Instruction  and  practice  would  be  required  on  the  oper¬ 
ational  equipment  after  practice  on  the  training  device  to  over¬ 
come  the  decrement. 


0 


Missing.  The  control  or  display  is  not  represented  at  all  in  the 
training  device. 


4.  Repeat  the  procedure  for  each  training  device  or  training 
device  concept  under  consideration. 

5.  For  each  training  device  or  device  concept  under  consid¬ 
eration  add  the  ratings  for  each  control  and  display  listed 
to  obtain  a  total  score. 

6.  Enter  the  total  score  for  each  device  in  Table  28. 

7.  Total  the  number  of  controls  and  displays  unique  to  a 
device  or  device  concept  in  addition  to  the  required  set 

of  controls  and  displays.  These  are  controls  and  displays 
unrelated  to  the  training  requirements  (see  previous 
footnote). 

8.  Derive  the  physical  similarity  index  value  by  dividing  the 
sum  of  physical  similarity  ratings  by  three  tiroes  the  number 
of  controls  and  displays  plus  the  number  of  controls  and 
displays  unique  to  a  device  or  device  concept. 

9.  Enter  the  physical  similarity  index  value  in  Table  2  8. 

A  separ:  te  physical  similarity  table  is  used  for  each  task.  How  faithfully 
the  training  device  represents  the  functioning  of  the  operational  equipment 
is  considered  below. 

Functional  Similarity- -The  functional  similarity  analysis  compares  the 
operator's  behavior  in  terms  of  the  information  flow  from  each  display  to 
the  operator,  and  from  the  operator  to  each  contro..  The  assessment 
is  made  in  terms  of  the  amount  of  information  transmitted  from  each 
display  to  each  control  and  the  type  of  information-processing  activity 
perform  -d  by  the  operator.  The  issue  is  not  the  physical  fidelity  of  a 
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control  or  display,  but  whether  the  operator  acts  on  the  same  amount  of 
information  in  the  same  way  in  both  operational  and  training  situations . 

Controls  and  displays  are  considered  in  conjunction  with  task  elements  to 
determine  the  type,  amount,  and  direction  of  information  flow  occurring 
during  task  performance.  Each  situation  in  which  a  display  transmits 
information  to  the  operator  (e.g.,  reads  it)  is  defined  as  a  stimulus  function, 
while  each  situation  in  which  the  operator  transmits  information  to  a  control 
(e.g.,  operates  it)  is  termed  a  response  function. 

In  each,  the  amount  of  information  may  be  estimated  by  the  number  of 
states  or  discrete  values  which  the  display  or  control  may  assume.  The 
functional  similarity  is  assessed  by  comparing  the  number  of  discrete 
values  or  states  in  the  training  situation  to  the  corresponding  number 
in  the  operational  situation.  The  range  of  values  may  be  partitioned  into 
four  major  levels:  1)  continuous  (essentially  unlimited),  2)  multivalued 
(some  discrete  value  greater  than  two),  3)  binary  (two  values),  or  4)  not 
represented.  Table  30  is  used  in  performing  functional  similarity 
analyses. 

1.  List  the  controls  and  displays  involved  in  task  performance  on  the 
operational  equipment  down  the  left  column  of  Table  30. 

2.  List  the  training  devices  or  training  device  concepts  to  be 
assessed  across  the  top  of  Table  30. 

3.  For  each  control  or  display  listed  in  the  left  column  of  Table  30 
note  how  well  it  is  functionally  represented  in  each  training 
device  under  consideration.  Base  your  ratings  of  functional 
similarity  on  the  four-point  scale  given  in  Table  31. 
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TABLE  30.  FUNCTIONAL  SD/1ILAJRITY  ANA 


NAME  OF  TASK 


REQUIRED 

TASK  CONTROLS  AND  DISPLAYS 

DEVICE  A 

DEVICE  8 

1. 

2. 

3- 

4. 

5. 

6. 

7. 

a. 

9. 

10. 

Sum  of  Functional  Similarity  Ratings 

Functional  Similarity  Index 

Functional  Similarity  =  _ Sum  of  Similarity  Ratings  for  Device _ 

index  3(^  of  Rq'd  Controls  &  Displays)  *  (2  of  Controls  &  Displays 


C.-li 


TABLE  31.  FUNCTIONAL  SIMILARITY  RATING  SCALE 


Rating  Definition 

3  Identi cal .  The  number  of  states  in  the  training  situation 

is  the  same  as  the  number  of  states  in  the  operational  setting. 

2  Similar.  The  number  of  states  in  the  training  situation  is  at 

least  Half  of  the  number  of  states  in  the  operational  setting. 

1  Dissimilar.  The  number  of  states  in  the  training  situation  is 

1  ess  Than"  half  of  the  number  of  states  in  the  operational  setting. 


0 


Missing.  The  control  or  display  is  not  represented  at  all  in 
the  training  device. 


•.  V  V'*  v  -  •  -  ’  ‘  *-  *  -V  ' 


4.  Repeat  the  procedure  for  each  training  device  or  training 
device  concept  under  consideration. 

5.  For  each  training  device  or  device  concept  under  consid¬ 
eration  add  the  ratings  for  each  control  and  display  listed 
to  obtain  a  total  score. 

6.  Enter  the  total  score  for  each  device  in  Table  30. 

7.  Total  the  number  of  controls  and  displays  unique  to  a  device 
or  device  concept  in  addition  to  the  required  set  of  controls 
and  displays,  if  any  such  controls  or  displays  exist  (see 
previous  footnote). 

8.  Derive  the  functional  similarity  index  value  by  dividing  the 
sum  of  functional  similarity  ratings  by  three  times  the  number 
of  controls  and  displays  plus  the  number  of  controls  and 
displays  unique  to  a  device  or  device  concept. 

9.  Enter  the  functional  similarity  index  value  in  Table  30. 

A  separate  functional  similarity  table  is  used  for  each  task. 

Skills  and  Knowledges  Requirements --This  analysis  assesses  the  skills 
and  knowledges  in  the  student's  repertory  before  training  and  compares  them 
to  the  skills  and  knowledges  required  for  successful  performance  of  the 
training  tasks.  Table  32  is  used  in  performing  this  analysis. 


1.  List  the  skills  and  knowledges  required  fo^  successful  task 
performance  down  the  left  column  of  Table  32. 
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Table  32.  Skill  and  Knowledge  Requirements  Analysis. 


NAME  OF  TASK 


REQUIRED  TASK  AFTER  BEFORE  DIFFERENCE 

SKILLS  AND  KNOWLEDGE  TRAINING  TRAINING  (AFTER  -  BEFORE) 


Skill  &  Knowledge  =  Total  Difference  Score _ 

Requirements  Index  4  (Number  of  Skills  &  Knowledge) 
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2.  For  each  skill  or  knowledge  listed  in  the  left  column  of 
Table  32,  estimate  how  "much"  of  the  skill  or  knowledge 
the  average  trainee  could  be  expected  to  have  upon  first 
exposure  to  the  training  device,  and  estimate  how  much  of 
the  skill  and  knowledge  he  must  possess  at  training's  com¬ 
pletion.  Use  the  two  scales  in  Tables  33  and  34. 

3.  Compute  the  differences  between  skill  and  knowledge  re¬ 
quirements  before  and  after  training  by  subtracting  the 
before -training  value  from  the  after -training  value  for  each 
skill  or  knowledge.  (Negative  differences  are  set  equal  to 
zero,  because  they  indicate  the  trainee  enters  with  more 
skill  than  necessary). 

4.  Add  the  difference  scores  ("after"  minus  "before"  ratings)  for 
the  skills  and  knowledges  listed  to  obtain  a  total  difference 
score. 

5.  Enter  the  total  difference  score  for  each  device  in  Table  32. 

6.  Derive  the  skill  and  knowledge  requirements  index  by 
dividing  the  total  difference  score  by  four  times  the  number 
of  skills  and  knowledges. 

7.  Enter  the  skill  and  knowledge  requirements  index  value 
in  T able  32  , 

A  separate  skill  and  knowledge  requirements  table  is  used  for  each  task. 


Task  Tr  lining  Difficulty-- A  task  training  difficulty  analysis  estimates  the 
difficulty  (in  terms  of  training  time)  of  training  soldiers  to  successfully 


TABLE  33.  RATING  SCALE  FOR  SKILLS  AND 
KNOWLEDGES  BEFORE  TRAINING 


Rating 

4 


I 

i 

! 

i 


3 


2 


1 


Defini tion 

Has  a  complete  understanding  of  the  subject  or  skill.  Can  do 
the  task  completely  and  accurately  without  supervision.  Has 
received  "skill"  training. 

Understands  the  subject  or  skill  to  be  performed.  Has  applied 
part  of  the  knowledge  or  skill  either  on  the  actual  job  or  a 
trainer.  Has  done  the  job  enough  times  to  make  sure  he  can  do  it, 
although  perhaps  only  with  close  supervision.  Has  had  "proce¬ 
dural"  training. 

Has  received  a  complete  briefing  on  the  subject  or  skill.  Can  use 
the  knowledge  or  skill  only  If  assisted  in  every  step  of  the 
operation.  Requires  much  more  training  and  experience.  Has 
received  "familiarization"  training  only. 

Has  only  limited  knowledge  of  this  subject  or  skill.  Has  not 
actually  used  the  Information  or  skill.  Cannot  be  expected  to 
perform.  Has  had  "orientation"  only. 


0 


No  experience,  training,  familiarity,  etc.,  with  this  skill  or 
knowledge.  '  Cannot  perform  a  task  requiring  this  skill  or 
knowledge. 


TABLE  34.  RATING  SCALE  FOR  SKILLS  AND 
KNOWLEDGES  AFTER  TRAINING 


Defini tion 

Should  have  a  complete  understanding  of  the  subject  or  be 
highly  skilled.  Is  able  to  perform  the  task  completely, 
accurately,  and  independently.  Has  had  "skill"  training. 

Should  have  an  understanding  of  the  subject  or  skill  to  be 
performed.  Has  applied  part  of  the  knowledge  or  skill  on  the 
actual  job  or  a  trainer.  Has  done  the  job  enough  times  to 
make  sure  he  can  do  it  although  perhaps  only  with  close  super¬ 
vision.  Needs  more  practice  under  supervision.  Has  had 
"procedural"  training. 

Should  have  received  a  complete  briefing  on  the  subject  or 
task.  Is  able  to  use-  the  knowledge  or  skit!  only  if  assis¬ 
ted  in  every  step  of  the  operation.  Requires  much  more  train¬ 
ing  and  experience  to  be  able  to  perform  the  task  Independently. 
Has  had  "familiarization"  training. 

Should  have  limited  knowledge  of  the  subject  or  skill.  Has 
not  actually  used  the  information.  Is  not  expected  to  perform 
the  task.  Has  completed  "orientation"  training. 

At  the  end  of  training,  the  trainee  should  have  no  experience 
or  training. 


uiiu  training  occurs  on  the  operational  equipment.  Use  Table  35  in  performing 
this  analysis. 

1.  List  the  task  elements  down  the  left  column  of  Table  35. 

2.  For  each  task  element  listed,  rate  the  amount  of  training  time 
for  surmounting  its  associated  deficit  relative  to  the  most  time- 
consuming  task  element.  Table  36  is  used. 

3.  Add  the  training  difficulty  ratings  for  the  task  elements  listed 
to  obtain  a  total  training  difficulty  score. 

4.  Enter  the  total  training  difficulty  score  in  Table  35. 

5.  Derive  the  task  training  difficulty  index  by  dividing  the  total 
training  difficulty  score  by  four  times  the  number  of  task 
elements. 

6.  Enter  the  task  training  difficulty  index  value  in  Table  35. 

A  separate  task  training  difficulty  table  is  used  for  each  task;  however,  the 
rating  scale  is  used  by  evaluating  training  time  for  each  task  element  rela¬ 
tive  to  the  most  time-consuming  element  for  all  tasks  in  the  current  analysis. 

Index  of  Training  Device  Effectiveness  - -The  five  analyses  just  completed 
are  used  to  derive  overall  indexes  of  effectiveness  for  each  training  device 
or  training  device  concept  under  consideration. 

•  Task  commonality 

»  Physical  similarity 

•  Functional  similarity 
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TABLE  35.  TASK  TRAINING  DIFFICULTY  ANALYSIS 


NAME  OF  TASK 


TABLE  36.  TASK  TRAINING  DIFFICULTY  RATING  SCALE 


Rating 

Definition 

4 

Requires  as  much  time  to  train  as  the  most 
time-consuming  task  element,  considering  all 
task  elements  for  all  tasks  Ln  the  current 
analysis 

3 

Requires  substantial  training  time,  but  less 
than  above 

2 

Requires  a  moderate  amount  of  training  time 
relative  to  the  most  time-  consuming  task  element 

1 

Requires  only  minimal  training  time  relative 
to  the  most  time-consuming  task  element 

0 

Requires  no  training  time 
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•  Skill  and  knowledge  requirements 


•  Task  training  difficulty 

The  task  commonality,  physical  similarity,  and  functional  similarity  analyses 
were  conducted  on  a  task-by-task  basis  for  each  training  device  or  training 
device  concept.  The  skill  and  knowledge  requirements  and  task  training 
difficulty  analyses  were  independent  of  any  particular  device  and  addressed 
specific  personnel  and  training  requirements.  The  following  procedure 
is  used  to  derive  task  level  training  effectiveness  indexes. 

1.  List  the  tasks  requiring  training  down  left  column  of 
Table  3  7. 

2.  Obtain  task  commonality,  physical  similarity,  and  func¬ 
tional  similarity  index  values  derived  in  previous  analyses 
(Tables  27,  28,  and  30)  for  each  task  listed  in  Table  37. 

3.  Enter  these  values  in  the  appropriate  columns  of  Table  37. 

4.  For  each  task,  add  the  listed  task  commonality,  physical 
similarity,  and  functional  similarity  scores  and  divide  by 
three.  The  obtained  value  indicates  the  degree  of  corre¬ 
spondence  between  the  operational  equipment  and  the 
particular  training  device  for  that  task. 


TABLE  37.  TRAINING  DEVICE  AND  OPERATIONAL 

EQUIPMENT  TASK  LEVEL  CORRESPONDENCE 


NAME  OF  DEVICE 


REQUIRED  TASKS 


TASK 

COMMONALITY 

(TC) 


PHYSICAL 

SIMILARITY 

(PS) 


FUNCTIONAL 

SIMILARITY 

(FS) 


TC  »  PS  FS 
3 


1. 


2. 


3. 


4. 


5. 


o. 


7. 


8. 


9. 


10. 


i 


The  next  analysis  summarizes  the  personnel  and  training  requirements 
malyses. 

1.  List  the  tasks  requiring  training  down  the  left  column  of  Table  38. 

2.  Obtain  the  skill  and  knowledge  requirements  and  task  training 
difficulty  index  values  derived  in  previous  analyses  (Tables  32  and 
35)  for  each  task  listed  in  Table  33. 

3.  Enter  these  values  in  the  appropriate  columns  of  Table  38. 

4.  For  each  task,  add  the  listed  skill  and  knowledge  requirements  and 
task  training  difficulty  index  values  and  divide  by  two.  The  ob¬ 
tained  value  indicates  the  extent  of  training  required  based  on 
skills  and  knowledge  requirements  and  abilities  of  the  typical 
trainee. 

The  final  analysis  derives  overall  training  device  effectiveness  indexes  for 
each 'device  or  device  concept  by  combining  the  task  level  training  device 
and  operational  equipment  correspondence  analysis  and  the  task  level 
personnel  and  training  requirements  analysis. 


1.  List  the  tasks  requiring  training  down  the  left  column  of  Table  39. 


2.  List  the  training  devices  or  training  device  concepts  to  be  assessed 
across  the  top  of  Table  39. 

3.  Obtain  the  TC+PS+FS  and  SKR+TTD  values  for  each  task  and  for 

3  2 

for  each  training  device  from  Tables  37  and  38  and  enter  them  in 
the  appropriate  columns  of  Table  39. 
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TASKS 


SKILL  &  KNOWLEDGE 

REQUIREMENTS 

(SKR) 


TASK  TRAINING 

DIFFICULTY 

(TTO) 


SKR  *  TTD 


10. 


Table  39.  Example  Training  Device  Effectiveness  Index. 


Pive  NA«u*\  <ouv\ 

•  i.5 

•7<? 

.5/ 

.831 

.7  <7 

.  toS 

.9  A 

.79 

.73 

2. 

Adjust  F've 

.M3 

.80 

.3¥ 

.70 

.30 

.56 

.83% 

.3o 

•  66 

(S)  Sum  of  Products 


Training  Device 
Effectiveness  Index 


Training  Device 
Effectiveness  ■ 
Index 


.83  fej 


i.m  hBBhi.54  wm 


.53  .76 


1.5*1 


.87 


a  x  _ S  af-B.wuir.wl  Iaala _ 

B  (#  of  Rq'd  Tasks)  ♦  (#  of  Tasks  Unique  to  Alternative  Device  Concept) 


‘  v .  -  .  A 


G-25 


PRODUCT 


4.  Multiply  the  TC  +  PS  +  FS  and  SKB  +  TTD  values  for  each 
task  and  enter  in  the  appropriate  column  labeled  product. 

5.  Add  ihe  product  values  for  all  tasks  and  enter  the  sum  in  the 
sum  of  products  location  for  that  device. 

6.  Add  the  SKH  +  TTD  values  for  each  task  and  enter  the  sum  in 

the  sum  of  SI^R  +  TTD  location  for  that  device. 

2 

7.  Divide  the  value  from  Step  5  ("A")  by  the  value  from  Step  6 
("B").  This  value  is  the  training  device  effectiveness  index 
for  that  training  device  concept. 

8.  When  comparing  alternative  existing  device  concepts,  an 
effectiveness  index  is  adjusted  by  multiplying  it  by: 

_ #  of  Required  Tasks  _ 

(#  of  Required  Tasks)  +  (#  of  Tasks  Unique  to  the  Alternative) 
This  factor  accounts  for  capabilities  within  an  existing  device 
that  are  not  required  for  the  device  under  consideration.  Cap¬ 
abilities  not  required  mean  additional  cost,  and  this  correction 
factor  adjusts  the  effectiveness  score  to  reflect  a  loss  of  effec¬ 
tiveness  due  to  unnecessary  cost.  Where  a  theoretical  device 
concept  is  being  considered,  there  probably  will  be  no  tasks 
unique  to  it  and  the  correction  factor  is  not  used. 

9.  Repeat  the  procedure  for  each  training  device  or  training 
device  concept  under  consideration. 


'~h«.  training  device  effectiveness  index  will  take  on  a  value  of  "1"  if  there 
is  a  perfect  correspondence  between  the  training  device  and  the  operational 
equipment.  The  effectiveness  index  will  take  on  a  value  of  "O"  when  there 
is  no  correspondence  between  the  device  and  the  operational  equipment. 

Example  af  Training  Device  Concept  Formulation 

The  following  example  illustrates  application  of  the  procedures  for  com¬ 
paring  alternative  training  device  concepts.  Three  devices  are  compared. 
For  purposes  of  this  example  assume  that  devices  A  and  B  are  existing 
devices  and  device  C  is  a  theoretical  training  device  concept.  Two  task 
training  requirements  (Table  40)  are  included  in  this  example.  Task 
commonality  (Tables  41  and  42),  physical  similarity  (Tables  43  and  44)  and 
functional  similarity  (Tables  45  and  46)  analyses  are  performed  for  each 
device  for  each  task.  Skill  and  knowledge  requirements  (Tables  47  and  48) 
and  task  training  difficulty  (Tables  49  and  50)  analyses  are  performed  for 
each  task.  Adjustments  are  made  for  task  elements,  controls,  and  displays 
unique  to  each  existing  device.  Tables  51  through  55  compute  effectiveness 
scores  for  each  device.  The  effectiveness  score  for  device  C  is  the  highest. 
Therefore,  the  theoretical  device  concept  should  be  developed  further. 

The  effectiveness  score  for  device  B,  an  already  existing  device,  is  lower; 
however,  its  score  is  sufficiently  high  to  conduct  further  evaluation.  Using 
an  existing  device  may  result  in  cost  savings  that  would  compensate  for  its 
lower  effectiveness  score.  Validation  testing  could  be  used  to  determine 
whether  device  B  or  device  C  is  more  appropriate. 


General 


SECTION  2  -  INSTITUTIONAL  TRAINING  CONCEPT 

The  data  required  to  develop  the 
institutional  training  costs  (ITC)  are  listed 
below.  This  format  is  provided  to  facilitate 
the  definitional  process  required  to  develop 
costs  for  Institutional  training  concepts  or 

courses  of  instruction.  The  requirements  herein  should  be  completed 
for  each  alternative  concept  under  consideration  and  for  each  course 
impacted. 


1.0  Course(s)  Impacted 

1.1  Modification  or  New  Course 

1.2  Describe  Changes  to  Current  Courses  of  Instruction 
2.0  Student  Load  Per  Class  Per  Year 

2.1  Average  Grade  of  Student 

2.2  Student  Source 

3.0  Class  Frequency  Per  Year 

3.1  Length  of  Class 

3.2  Fiscal  Year  of  Course  Start 

4.0  Instructor  Requirements  Per  Affected  Subcourse(s) 

5.0  Support  Personnel  Requirements 

6.0  Expended  Equipment  Per  Class  (Subcourse) 

7.0  Non-expended  Equipment  Used  and  Usage  Rate  Per  Subcourse 
8.0  Exportable  Training  Used 

8.1  Exportable  Software 

8.2  Training  Teams 

8.3  Exportable  Hardware 
9.0  Fac 11 Ity  Requirements 

Exampl  e 

An  outline  and  an  example  of  a  hypothetical  institutional 
training  concept  is  shown  below. 


SAMPLE:  INSTITUTIONAL  TRAINING  CONCEPT  (HYPOTHETICAL) 


2.2  Student  Source 


Students  in  the  11C10  course  are  in  AIT  phase  of  OSUT. 

3.0  Class  Frequency  Per  Year 

Year 


I  1 

I  •  • 

...  20 

20  17 

12 

12 

Length  of  Class 

Current 

Modified 

Full  Course 

10  weeks 

9  weeks 

Crew  Drill  Subcourse  3  weeks  2  weeks 

3.2  Fiscal  Year  of  Course  Start 

The  modified  11C10  course  utilizing  the  X99  MCT  will  be 
feasible  In  FY  87. 

4.0  Instructor  Requirments  Per  Affected  Subcourse(s) 

Crew  Drill  subcourse  instructors  per  Class: 


Current 

Modified 

11C40  E7 

160  Manhours 

110  Manhours 

11C30  E6 

160  Manhours 

110  Manhours 

1  IB  03 

80  Manhours 

50  Manhours 

118  04 

80  Manhours 

50  Manhours 
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5.0  Support  Personnel  Requirements 


The  modified  subcourse  will  require  no  support  personnel; 
however,  since  the  simulated  crew  drill  replaces  several  live 
fire  exercises,  the  following  personnel,  currently  required, 
will  be  freed  for  other  purposes: 

Current  Per  Class 

11C 30  E6  480  Manhours 

11C 20  E5  960  Manhours 

11C 20  E4  1920  Manhours 

6.0  Expended  Equipment  Per  Class  {Subcourse) 


Current 

Modified 

Cartridge  M374 

800  (250) 

550  (0) 

Cartridge  M57 

200  {  25) 

175  (0) 

Cartridge  M301 

50  (  0) 

50  (0) 

Cartridge  M362 

un 

in 

CO 

80  (0) 

Cartridge  M375 

100  (100) 

0  (0) 

Soldiers  Manual  1 1C 

50 

50 

Non-expended  Equipment 

Used  Per  Subcourse 

Current 

Modified 

M151  1/4  T 

160  mil  es 

10  miles 

26  Passenger  Bus 

320  miles 

20  miles 

M29A1  Mortar 

380  RDS 

0  RDS 

M35  2  1/2  T 

160  mil  es 

C  mi  1  es 

PU-619  Gen 

80  hours 

0  hours 

Instructors  Manual  1 1C 


10 


10 


Instructors  Manual  X99  0  10 

8.0  Exportable  Training  Used 
None 

8. 1  Exportable  Software 

None.  (This  would  be  applicable  to  training  packages  sent  to 
the  field  such  as  manuals,  training  films,  circulars, 
bulletins,  cassettes,  etc.) 

8.2  Training  Teams 

None.  (If  teams  are  sent  from  the  institution  to  the  units  to 
provide  training  on  new  equipment,  doctrine,  or  tactics,  this 
section  will  reflect  team  composition,  schedule,  units  trained 
personnel  trained,  and  assets  utilized  during  training.) 

8.3  Exportable  Hardware 

None.  (This  section  will  describe  quantities  and  types  of 
hardware  to  be  provided  to  units  by  institutions  to  support 
training  requirements  for  the  alternative.) 

9.0  Facility  Requirements 


Institution 

Unit 


Classroom  Ranges 

3000  Sq  Ft* 

Permanent  brick  None 
None  None 


*2,000  Sq  Ft  exist;  1,000  Sq  Ft  require  construction. 


SECTION  3  -  UNIT  TRAINING  CONCEPT 


General 

The  unit  training  concept  should  also  be  defined  by  the  proponent 
and  is  outlined  below.  This  format  is  provided  as  a  means  of  capturing 
pertinent  unit  training  factors  for  use  in  calculating  training  costs. 
The  outline  herein  should  be  used  to  define  a  generic  unit  training 
routine  for  each  training  alternative  of  interest. 

1.0  Training  Alternatives 

2.0  Training  Events  (This  section  to  be  completed  for  ea'-h 
Generic  Training  Event) 

2.1.1  Training  Event  #1  (Title,  Soldiers  Manual,  or  ARTE 
Reference) 

2.1.2  Description  of  Training  Conducted 

2.1.3  Level  of  Training  Aggregation  (Squad,  Pit,  Bn,  Olvlslon) 

2.1.4  Instructor  Personnel  Required  and  Parent  Unit  (Organic  to 
Level  of  Training  Conducted) 

2.1.5  Personnel  Trained  (Quantity,  Grade,  MOS) 

2.1.6  Location  of  Training,  Distance  from  Unit  Area,  Means  of 
Transportation 

2.1.7  Equipment  Utilized  and  Rate  Per  Event  (Miles,  Hours,  etc.) 

2.1.8  Facility  Requirements  (Classrooms,  Ranges,  Motor  Pool 
8uildings,  etc.) 
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2.1.9  Time  Requirements  Per  Training  Event 

2.1.10  Frequency  of  Training  Event  Repetition  (Times  Per  Month, 
Quarter,  or  Year) 

2.1.11  Support  Personnel  and  Equipment  Requirements  (Personnel 
Other  than  Students  and  Instructors) 

2.1.12  TDY  Requirements  for  Training  Event  #1 

2.2.1  Training  Event  #2  (Title) 

2.2.2  Description  of  Training  Conducted 


2.N. 12  TDY  Requirements  for  Training  Event  #N 

Exampl e 

The  following  is  an  example  of  a  hypothetical  unit  training 
concept. 

SAMPLE:  UNIT  TRAINING  CONCEPT  (HYPOTHETICAL) 

1.0  Training  Alternative 

The  unit  training  concept  definition  is  the  program  which  is 
anticipated  to  be  used  with  the  intensified  simulation 
training  for  REDEYE  gunners  (hypothetical). 

2.0  Training  Events  #1  through  #15 

2.1.1  Training  Event  #1 

Engage  Hostile  Aircraft;  Task  1057,  FW  44-1 6P 


2.1.2  Description  of  Training  Conducted 

Gunners  engage  simulated  hostile  aircraft  in  a  combat 
environment  utilizing  the  moving  target  simulator  (MTS). 

Each  gunner  will  track  25  aircraft,  scoring  simulated  hits 
on  no  fewer  than  80  percent  of  engageable  targets. 

2.1.3  Level  of  Training  Aggregation 

Training  event  will  be  conducted  at  the  REDEYE  section  level 
for  each  battalion. 

2.1.4  Instructor  Personnel  Required  and  Parent  Unit 

Instructors  will  be  those  organic  to  the  local  or  supporting 
MTS  and  will  not  be  part-of  the  unit  trained.  In  general, 
the  instructor  requirements  will  consist  of  one  E6  16P30  per 
section  trained. 

2.1.5  Personnel  Trained  (per  battalion) 

16P10,  E4,  5  each 

2.1.6  Location  of  Training,  Distance  from  Unit  Area,  Means  of 
Transportation 

MTS  facilities  are  located  in  each  major  troop  concentration 
geographic  area  such  as  Ft  Hood,  Ft  Riley,  Ft  Bragg, 

Ft  Bliss,-  7th  Army  ATC,  Ft  Lewis,  and  Camp  Casey.  The 
average  REDEYE  section  would  be  required  to  travel  approx¬ 
imately  75  miles  one-way  to  reach  the  MTS  facility. 

Organic  section  vehicles  would  be  used  to  and  from  the  MTS. 
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2.1.7  Equipment  Utilized  and  Rate  Per  Event 

MTS  8  Hours  Per  Event 

Tracking  Head  Trainer  16  Hours 

1/4T  Truck  300  Miles  Per  Event 

2.1.8  Facil  ity  Requirements 
MTS  Screen 

2.1.9  Time  Requirements  Per  Training  Event 
8  Hours  Per  Section 

2.1.10  Frequency  of  Training  Repetition 

The  engagement  simulation  training  will  be  conducted  at 
least  monthly  for  each  REDEYE  section  or  12  times  annually 
for  each  gunner. 

2.1.11  Support  Personnel  and  Equipment  Requirements 

GS-9  Electronics  Technician  3  Hours  Per  Training  Event 

2.1.12  TDY  Requirements  Per  Training  Event  #1 

Three  REDEYE  sections  are  located  in  isolated  areas  from 
the  nearest  MTS  facility  and  will  require  three  days  per 
diem  pay  for  each  training  event  plus  commercial  air 
transportation  costs  of  approximately  $135  per  man  per 


2.2.1  Training  Event  #2 


Visual  aircraft  recognition;  Task  1040,  FM  44-16P 


2.N.10  Frequency  of  Training  Event  Repetition 

Task  1061,  determination  of  aircraft  category  for  REDEYE 
ranging  will  be  conducted  weekly  per  section  using  the 
modified  GOAR  kit. 


2.N. 12  TOY  Requirements  Per  Training  Event  #N 
None 

SECTION  4  -  TRAINING  SYSTEM  REQUIREMENTS  SPECIFICATION  (TSRS) 

General 

This  format  is  provided  to  facilitate  the  definitional  process 
required  to  develop  costs  for  training  devices  or  systems.  It  is 
intended  as  a  means  of  standardizing  required  input  to  preclude  both  too 
little  and  too  detailed  input. 


1.0  Hardware  Description 

1.1  Physical  Characteristics  (dimensions,  weight,  space 

requirements) 

1.1.1  Functional  Characteri sties  (task  flow,  operating  layout, 
input/output  description) 

1.2  Power  Requirements  (kw/hr,  volts,  ac/dc) 

1.3  Calibration  Requirements  (level,  location,  frequency)  • 

1.4  Maintenance  hours  per  unit  of  operation  (hr/yr,  hr/mile, 
hr/ round,  etc.) 

1.4.1  Maintenance  Level  and  Location 

1.5  Depot  Overhaul  Frequency  (overhauls/time  period) 

1.6  Computer  Requirments 

1.6.1  Type  (general  or  specialized,  time  share  of  dedicated, 
available  or  required) 

1.6.2  Storage  Capacity  (main  core  and  peripheral) 

1.6.3  Compiler  (available  or  not  available) 

1.6.4  Word  Size  (bits  per  word) 

1.6.5  Software  Requirements 

1.7  Hardware  Replaced  and  Replacement  Ratio 

1.8  Personnel  Replaced 

1.9  Projected  Operational  Life 


2.0  Employment  Concept 


2.1  Annual  Operating  Factor  (hours,  miles,  rounds,  use  per 
training  event/ frequency  of  events) 

2.2  Operators  Required  (grade,  MOS,  quantity) 

2.3  level  of  Use 

2.3.1  Institutional  (BCT,  AIT,  NCOES,  OBC,  AOC,  etc.) 

2.3.2  Unit  (individual,  squad,  platoon,  etc.) 

2.4  Basis  of  Issue 

2.4.1  Institutions 

2.4.2  Units  (Active  and  Reserve) 

2.5  Float  Requirement  Ratio  (float  devices  per  each  operational 
device) 


The  TSRS  provides  the  minimum  Information  required  about  the  system 
before  cost  analysis  can  take  place.  The  source  of  this  Information 
would  primarily  be  the  proponent  with  assistance  from  DARCCM.  The 
general  intent  of  the  TSRS  Is  to  define  the  system  in  such  detail  as 
to  a  allow  the  analyst  to  define  and  direct  his  cost  effort. 

Exampl  e 

An  outline  and  example  of  a  hypothetical  TSRS  is  shown  below. 

SAMPLE:  TRAINING  SYSTEM  REQUIREMENTS  SPECIFICATION  (HYPOTHETICAL) 
SYSTEM:  Fire  Control  Simulator  BT33 
1.0  Hardware  Description 
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1.1 


Physical  Characteristics 

Screen  Size:  6m  x  1.5m 

Central  Unit:  4m  x  lm  x  1.5m 

Slide  Projectors:  .5m  x  .5m  ,2m;  8mm  (4  ea) 

Space  Requirements:  3000  sq  ft  classroom 

Student  Capacity:  50 


1.1.1  Functional  Characteristics 

The  BT33  simulates  artillery  fire  and  adjustment  of  fires  on 
stationary  and  moving  targets.  The  battlefield  panorama 
Including  the  target  area  is  projected  on  the  screen  with  a 
slide  projector.  The  observer  (students)  can  see  the 
projected  area  from  the  simulated  OP.  The  instructor 
Indicates  the  target  display  projector.  The  students 
observe  the  target  and,  using  simulated  communications 
procedures,  make  inputs  to  the  device  operator  (simulated 
fire  distribution  center).  The  operator  inputs  data  to 
Implement  the  student's  calls  for  fire.  The  central  unit 
calculates  the  position  of  the  bursts  in  the  furtherest 
area  and  projects  the  burst  symbols  onto  the  screen  with 
the  burst  projectors.  The  student  may  then  adjust  fire, 
lift  fire.  Increase  firei  or  report  damage  assessment. 


1.2  Power  Requirements 

220  VAC,  50-60  Hz,  1  kw/hr 


1.3  Cal  ibratlon  Requirements 

System  is  aligned  quarterly,  total  manpower  requirements 
equal  one  manhour  per  quarter.  Cal ibration  performed  by 
local  unit  at  user  location,  level  c. 


1.4  Maintenance  Hours  Per  Unit  of  Operation 

The  BT33  required  an  average  of  one  hour  of  direct 
maintenance  for  every  40  hours  of  operation 


1.4.1  Maintenance  Level  and  Location 

Organizational,  DS,  and  GS  maintenance  are  authroized  to  be 
conducted  by  the  qualified  operator  on  station. 


1.5  Depot  Overhaul  Frequency 

The  BT33  will  require  depot  overhaul  every  10  years.  Depot 
overhaul  will  require  the  BT33  to  be  transported  to  Red 
River  Depot,  Texas. 


1.6  Computer  Requirements 


1.6.1  Type 

Specialized.  The  BT33  is  complete  with  Its  own  processing 
unit  which  is  dedicated. 


1.6.2  Storage  Capacity 

The  core  memory  will  store  4,096  words,  each  containing  16 
bits. 
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1.6.3  Compilers 


None 


1.6.4  Word  Size 
16  bits 


1.6.5  Software  Requirements 

The  BT33  contains  8mm  film  cassettes  to  simulate  targets 
plus  pre-programmed  fire  mission  commands  controlled  by  the 
operator  console. 


1.7  Hardware  Replaced  and  Replacement  Ratio 

In  the  Institutional  environment  the  BT33  will  replace  four 
puffboards  (terrain  mock-ups)  and  possibly  replace  live  fire 
exercises  if  no  effectiveness  degradation  results.  The 
replacement  of  the  puffboard  will  enhance  realism  while 
the  replacement  of  the  live  fire  exercises  could  result  in 
cost  savings. 


1.8  Personnel  Replaced 
None 


1.9  Projected  Operational  Life 

Twenty  years  with  one  depot  overhaul. 
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2.0 


Employment  Concept 


2.1  Annual  Operating  Factor 

Institution:  (24  hr  per  week  -  52  weeks)  1,248  hr  per  year 
Unit:  (80  hr  per  month  -  12  months)  960  hr  per  year 

2.2  Operators  Required 

One  per  device  minimum 
Institution:  E7  13E40 
Unit:  03  13F 


2.3  Level  of  Use 

2.3.1  Institutional 

OBC,  Advanced  NCOES  (13E) 

2.3.2  Unit 

DIVARTY  FIST;  maneuver  unit  leaders  courses 

2.4  Basis  of  Issue 

2.4.1  Institutional 
U SAFAS:  2 

2.4.2  Units 

2.5  Float  Requirement  Ratio 
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SECTION  5  -  LIFE  CYCLE  COST  ESTIMATE  (LCCE) 


The  LCCE  considers  all  costs  incurred  during  the  projected  life  of 
the  training  system  and/or  subsystem.  The  LCCE  includes  the  cost  to 
acquire,  operate,  and  maintain  the  training  hardware  (device)  over  its 
useful  life..  Materiel  system  life  cycle  cost  includes  all  costs 
associated  with  the  three  life  cycle  phases:  research  and  development 
(R&D),  investment,  and  operations.  Depicted  in  Figure  1-1  is  a  profile 
of  the  life  cycle  of  a  materiel  system  and  cost  phases. 


APPENDIX  I 


HRTES  PROCEDURES  FOR  IDENTIFYING  OT  TEST'  ISSUES 


This  appendix  describes  HRTES  procedures  for  developing  OT 
test  issues.  The  description  of  these  procedures  was  taken 
from  the  HRTES  Prototype  Handbook,  pages  123  to  175. 


HUMAN  PERFORMANCE  MEASURES 


In  this  section,  you  will  identify  the  performance  measures  to  take  during 
the  operational  test.  Read  the  general  section  on  performance  measures 
(time  and  accuracy).  These  will  reference  the  human  performance  functions 
you  gathered  from  section  6.  Afte1'  completing  this  section  you  should 
have  developed  specific  measures  of  human  performance  which  can  be  used  in 
the  operational  test. 


NEXT:  Turn  to  Test  Conditions  page  i-i3). 


HUMAN  PERFORMANCE  MEASURE  1 
TIME-DURATION  OF  HUMAN  PERFORMANCE  FUNCTION 

RATIONALE 

This  is  the=mea-sure  of  the  length  of  time  between  the  presentation 
of  that  cue  which  marts  ’the  initiation  of  a  Human  Performance  Function  (HPF) 
and  the  completion  of  that  HPF,  or  the  cessation  of  HPF  behavior,  in  the 
absence  of  completion.  This  measure  incorporates  reaction  time  according 
to  the  nature  of  the  HPF^s  measured  in  HRTES.  If  an  HPF  were  "Target 
Detection",  then  the  duration  of  that  HPF  would  be  identical  to  reaction 
time.  Therefore,  for  the  purposes  of  simplification  and  clarity,  HPF's 
are  so  structured  that  duration  is  the  only  time  measure  required. 

The  length  of  time  it  takes  to  perform  an  HPF  is  important  for  two 
reasons:  -  - 

(1)  Many  HPF's  can  only  be  accomplished  successfully  as  a 
function  of  time.  That  is,  if  a  given  HPF  were  not  completed 
within  a  specified  period  of  time,  it  would,  by  definition, 
be  considered  a  failure.  In  addition,  the  faster  a  given 
HPF  were  completed,  the  greater  would  be  the  level  of  its 
success. 

(2)  Comparable  systems  frequently  have  HPF's  in  common.  Time  to 
perform  an  HPF  is  one  measure  of  the  difficulty  of  that  HPF. 

The  larger  the  number  of  more  difficult  HPF's  a  given  system 
required, ^ff^ess  desirable  is  that  system,  all  other  factors 
being  equaftr^Therefore ,  time  to  perform  may  be  used  to 
differentiate  the  relative  desirabilities  of  competing  systems. 


METHODOLOGY 


I.  Identify  the  cue  which  initiates  HPF  behavior. 

1.1  In  the  first  HPF  of  a  series,  this  cue  is  normally  the  introduction 
of  a  new  stimulus,  such  as: 

1.1.1  visual  target 

1.1.2  auditory  signal 

1.1.3  command 

1.2  In  later  HPF's  in  a  series,  the  cue  is  often  the  completion  of 
the  previous  HPF. 

1.3  Example  -  Target  acquisition: 

1.3.1  target  detection-presentation  of  target 

1.3.2  target  identification-completion  of  identification  or,  in 
some  cases,  target-presentation 

1.3.3  target  location-presentation  of  target 

1.3.4  aim/lock  on-completion  of  target  location 

Frequently,  the  HPF's  may  not  take  place  in  as  linear  a  manner  as  shown 
above.  For  the  purpose  of  the  OT,  it  is  necessary  that  recordable  initiation 
cues  either  be  determined  or  be  artificially  introduced  prior  to  the 
start  of  actual  testing. 


Determine  the  smallest  time  interval  required  for  measurement  of  the 
given  HPF. 

2.1  For  HPF's  which  consist  entirely  of  reaction  to  a  presented 
stimulus,  one  tenth  of  a  second  is  the  smallest  time  interval 
required,  unless  otherwise  stated. 

2.2  For  HPF’s  which  consist  of  motor  behavior  or  some  combination  of 
motor  and  intellectual  behavior,  the  smallest  time  interval  will 
vary  with  the  HPF.  A  reasonable  rule  of  thumb  is  that  the 
smallest  time  interval  required  is  one-tenth  of  the  interval 
used  in  stating  the  criterion. 

Identify  the  incident  which  informs  you  that  either  the  HPF  has  been 
completed,  or  that  HPF  behavior  has  ceased. 

3.1  In  the  case  of  HPF  completion,  the  end  point  is  defined  in 
advance.  Since  this  end  point  may  be  entirely  intellectual 
and  not  observable,  provisions  may  be  required  to  aid  in  end 
point  recognition. 

3.1.1  target  detection  -  either  operator  switch  closing,  or 
vocal  signal  upon  detection,  depending  on  the  amount 
of  money  and  time  available. 

3.1.2  target  identification  -  vocal  signal  of  the  nature  of  the 
target,  or  prestructured  switch  coding  indicating  nature 
of  target. 

3.1.3  target  location  -  vocal  signal  of  target  coordinates, 
measured  from  initiation  of  vocal  response. 
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3.1.4  weapon  aiming  -  completion  of  each  aiming  incident, 
measured  to  completion  -  not  initiation  of  aiming. 

3.2  In  the  case  of  the  cessation  of  HPF  behavior,  in  the  absence 
of  completion,  three  general  methods  are  available: 

3.2.1  vocal  signal 

3.2.2  switch  closing 

3.2.3  observation  -  This  is  the  most  questionable  technique, 
but  in  the  confusion  which  often  accompanies  cessation 
without  completion,  it  may  be  necessary  to  use  this 
approach  to  avoid  complete  loss  of  data. 

4.  Select  conditions  in  which  HPF  is  to  be  run. 

NOTE:  It  may  be  desirable  to  append  the  most  important  of  these 
conditions  directly  to  the  selected  SPI,  thereby  creating  a 
composite  SPI  which  emphasizes  that  condition.  See  appropriate 
Note  accompanying  the  SPI's  for  further  information  on  this 
subject. 

4.1  There  are  ten  classes  of  conditions: 

4.1.1  weather  4.1.6  ground  surface 

4.1.2  light  4.1.7  ground  slope 

4.1.3  target  4.1.8  obstacles/concealment 

4.1.4  personnel  4.1.9  tactical 

4.1.5  operational  4.1.10  training 

For  a  more  detailed  analyis  of  these  classes  of  conditions,  and  a  system 
for  ranking  them  for  criticality,  see  the  Condition  Ranking  System  on 
page 
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Determine  the  number  of  subjects  to  be  run  in  the  test  to  collect  the 
measure. 

t 

Determine  the  number  of  trials  to  run  per  subject. 

Determine  the  method  for  collecting  the  data. 

7.1  There  are  three  basic  problems  which  impact  the  collection  of 
data: 

7.1.1  How  much  money  is  available? 

7.1.2  How  much  time  is  available  for  preparation? 

7.1.3  How  much  space  is  available  in  the  system? 

7.2  These  three  problems  interact  with  each  other.  If  neither  space, 
time,  nor  money  is  a  serious  issue,  it  Is  best  to: 

7.2.1  include  a  trained  observer  with  each  system  in  the  test 

7.2.2  fully  instrument  each  system  in  the  test  so  that  HPF  cues 
indicating  initiation  and  completion,  as  well  as  time 
measures,  would  be  automatically  recorded. 

7.3  If  there  is  no  room  for  an  observer,  instrumentation  is  a  firm 
requirement. 

7.4  If  there  is  room  for  an  observer,  but  not  sufficient  funding  for 
full  instrumentation,  it  is  best  to  train  the  observer  as  fully 
as  possible. 

7.5  If  there  is  room  but  insufficient  funding  for  full  instrumentation 
and  insufficient  time  for  complete  observer  training,  it  is  best 
to  select  those  individuals  to  be  observers  who  are  either 
experienced  in  operational  testing,  or  who  learn  quickly  and  are 
adaptable. 


7.6  One  factor  is  clear:  one  must  collect  data.  The  more  objective 
the  method  of  collecting  the  data,  the  better  quality  it  is  likely 
to  be.  It  is  preferable  to  minimize  reliance  on  operator  reports 
for  objective  data  collection.  Therefore,  systems  to  be 
investigated  should  be  instrumented  and/or  supplied  with  observers 


HUMAN  PERFORMANCE  MEASURE  2 
ACCURACY  OF  HUMAN  PERFORMANCE  FUNCTIONS 


RATIONALE 


This  is  a  group  of  measures  relating  to  the  accuracy  with  which 
personnel  carry  out  Human  Performance  Functions  (HPF's).  An  HPF  may  be 
either  omitted  or  performed.  If  it  is  performed,  the  accuracy  of  its 
performance  is  variable.  Based  on  judgement,  one  point  on  this  continuum 
may  be  selected  as  the  lowest  level  of  acceptable  performance.  This 
point  becomes  the  criterion.  All  performance  superior  to  the  criterion 
is  desirable,  but  may  cost  more  to  achieve  than  is  warranted  by  system 
requirements.  All  performance  inferior  to  the  criterion  is  undesirable, 
and  that  undesirability  increases  as  the  level  of  accuracy  decreases. 

Determination  of  HPF  accuracy  serves  three  functions: 

(1)  Analysis  of  System  Performance:  It  provides  important  data 
for  the  analysis  of  total  system  performance,  leading  to  an 
understanding  of  the  reasons  that  a  system  has  not  achieved 
all  expected  performance  criteria. 

(2)  Analysis  of  Human  Performance:  It  provides  data  leading  to 
an  understanding  of  the  bases  of  human  performance  inaccuracy 
and  the  steps  needed  to  reduce  that  inaccuracy. 

(3)  Human  Performance  Reliability:  It  provides  data  required 
for  the  eventual  projection  of  human  performance  reliability 
in  system  operation. 
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METHODOLOGY 


1.  Determine  the  criticality  of  the  problem  in  question. 

1.1  It  would  be  iesirable  if  there  were  some  objective  method  for 
determining  the  criticality  of  problems.  Unfortunately,  this 
can  be  done  only  through  the  use  of  judgment.  It  is  clear  that 
some  problems  are  more  critical  than  are  others,  and  that  this 
difference  may  provide  some  increased  meaning  to  the  output  of 
the  OT.  The  following  rating  scale  can  be  photocopies  and  given 
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2.1  Measures  of  HPF  accuracy  deal  with 

(1)  problems  in  performance 

(2)  performance  effectiveness 

(3)  effects  of  inadequate  performance.  They  include  evaluative 
measures  which  are  applicable  to  other  HPF  measure  areas 
and  will  be  so  noted. 

2.1.1  number  of  problems/deviations  from  accurate  performance 

2.1.2  size  of  problems/deviations 

2.1.3  direction  of  problems/deviations 

NOTE:  The  following  4  measures  include  omissions,  sequence  problems,  and 
accidents  with  accuracy  measures: 

2.1.4  system  downtime  due  to  all  problems 

2.1.5  percentage  of  HPF’s  run  without  problems 

2.1.6  percentage  of  subcriterion  trials  of  HPF's 

2.1.7  number  of  repetitions  required  to  perform  HPF  to 
criterion 

2.2  Measures  of  omission  refer  to  the  absence  of  either  an  entire  HPF,  or 

one  or  more  of  its  elements.  Elements  are. those  finer  pieces  of 

behavior  which  are  performed  to  carry  out  the  HPF. 

2.2.1  number  of  HPF's  totally  omitted  for  a  given  System 
Performance  Issue 

2.2.2  number  of  HPF  elements  omitted  for  a  given  HPF 

2.2.3  number  of  omitted  HPF  elements  for  a  given  HPF  by  category 

2.2.3. 1  cueing  elements 

2. 2. 3. 2  perception  elements 

2. 2. 3. 3  cognitive  elements 

2. 2. 3. 4  motor  elements 


2.3  Measures  of  sequence  refer  to  the  order  in  which  HPF's  and  HPF 
elements  are  performed.  In  some  cases,  sequence  is  a  very  important 
measure,  but  in  others  it  will  be  either  meaningless  or  essentially 
impossible  to  collect.  If  you  are  considering  collecting  sequence 
measures,  carefully  consider  if  there  really  is  a  specific  sequence 
in  which  the  HPF  or  its  elements  are  to  be  performed. 

2.3.1  number  of  HPF's  performed  out  of  sequence  (percentage  of 
HPF's  performed  out  of  sequence  for  a  given  System  Performance 
Issue  is  also  useful) 

2.3.2  number  of  HPF  elements,  for  a  given  HPF,  performed  out  of 
sequence  (percentage  of  HPF  elements  performed  out  of 
sequence  is  also  useful) 

2.3.3  direction  of  sequence  shift 

2.3.4  number  and  sequence  shift  of  HPF  elements  by  category: 

2.3.4. 1  cueing  elements 

2. 3. 4. 2  perception  elements 

2. 3. 4. 3  cognitive  elements 

2. 3. 4. 4  motor  elements 

2.4  Measures  of  accidents  refer  to  both  personnel  and  equipment 
accidents.. Accidents  to  personnel  need  not  involve  injury.  They 
do  involve  occurrences  which  lead  to  injuries.  Accidents  to 
materiel  need  not  involve  damage.  They  involve  incidents  such  as 
dropping,  striking,  hitting,  etc.,  which  could  be  expected  to 
produce  damage.  Critical  incidents  are  those  occurrences  in  which 
neither  a  personnel  nor  materiel  accident  takes  place,  but  in  which 
such  an  accident  almost  takes  place. 


2.4.1  number  of  accidents  associated  with  HPF  (personnel  and 
materiel ) 

2.4.2  number  of  injuries  associated  with  HPF 

2.4.3  number  of  man-hours  lost,  associated  with  HPF 

2.4.4  number  of  materiel  accidents  resulting  in  damage,  associated 
with  HPF 

2.4.5  cost  of  replacement/repair  of  materiel  involved  in 
accident  associated  with  HPF 

2.4.6  number  of  man-hours  lost  due  to  materiel  accident  associated 
with  HPF 

2.4.7  number  of  critical  incidents  for  personnel  and  for  materiel 
associated  with  HPF 

3.  Select  conditions  in  which  HPF  is  to  be  conducted. 

NOTE:  It  may  be  desirable  to  append  the  most  important  of  these  conditions 
directly  to  the  selected  SPI,  thereby  creating  a  composite  SPI  which 
emphasizes  that  condition.  See  appropriate  Note  accompanying  the  SPI's  for 
further  information  on  this  subject. 


3.1  There  are  ten  classes  of  conditions; 


3.1.1  training 

3.1.2  weather 

3.1.3  light 

3.1.4  ground  surface 

3.1.5  ground  slope 

3.1.6  obstacles/concealment 

3.1.7  target 

3.1.8  personnel 

3.1.9  operational 
3.110  tactical 

For  a  more  detailed  analysis  of  these  classes  and  a  system  for  ranking  them, 
see  the  Condition  Ranking  System  on  page 

3.2  It  is  assumed  that  0T  will  exercise  the  system  under  optimum 

versions  of  probable  conditions.  It  is  also  desirable  for  probable 
worst  case  conditions  to  be  exercised  in  0T.  If  one  or  more  of 
the  conditions  has  been  selected  for  inclusion  in  the  System 
Performance  Issue,  one  should  still  examine  the  others.  Inclusion 
of  a  condition  directly  into  a  System  Performance  Issue  does  not 
mean  that  no  other  condition  is  applied;  it  means  that  this 
condition  is  of  preeminent  importance  and  must  be  tested. 

4.  Determine  the  method  for  collecting  the  data. 

4.1  There  are  three  general  methods  for  collecting  data: 

4.1.1  observation 

4.1.2  instrumentation 

4.1.3  debriefing 

4.2  Observation  requires  an  observer  who  has  some  level  of  training 
and  space/time  in  which  to  work,  as  well  as  some  sort  of  structured 
data  collection  form. 


4.3  Instrumentation  requires  an  understanding  of  the  nature  of 
the  data  to  be  collected,  a  piece  of  appropriate  performance 
measuring  equipment,  a  means  for  activating  it  at  the  right 
times,  a  power  source,  and  a  place  to  put  it.  Debriefing 
requires  a  subject  with  some  information,  a  structured  debriefing 
form,  a  place/time  for  the  debriefing  and,  in  some  cases,  a 
debriefing  individual  who  has  some  experience  and/or  ability 

in  the  area. 

4.4  There  are  three  basic  problems  which  impact  the  collection  of 
data: 

4.4.1  How  much  money  is  available? 

4.4.2  How  much  time  for  preparation  is  available? 

4.4.3  How  much  space,  in  the  system,  is  available? 

4.5  These  three  problems  interact  with  each  other.  If  neither 
space,  time,  nor  money  is  a  serious  issue,  it  is  best  to: 

4.5.1  include  a  trained  observer  with  each  system  in  the 
test 

4.5.2  fully  instrument  each  system  in  the  test  so  that  HPF 
cues  indicating  initiation  and  completion,  as  well 
as  time  measures,  would  be  automatically  recorded. 

4.6  If  there  is  no  room  for  an  observer,  instrumentation  is  a 
firm  requirement. 


4.7  If  there  is  room  for  an  observer,  but  not  sufficient 
funding  for  full  instrumentation,  it  is  best  to  train 
the  observer  as  fully  as  possible. 

4.8  If  there  is  room,  but  neither  sufficient  funding  for  full 
instrumentation  nor  sufficient  time  for  complete  observer 
training,  it  is  best  to  select  those  individuals  to  be 
observers  who  are  i  ither  experienced  in  operational 
testing  or  who  learn  quickly  and  are  adaotable. 

4.9  It  is  necessary  that  observers/instrumentation  be  so 
placed  and  trained  that  they  are  able  to  record  the 
following  sorts  of  information  about  HPF  behavior: 

4.9.1  What  was  the  cueing  stimulus?  Did  either  incorrect 
cueing,  too  general  a  cueing  stimulus,  too  heavily 
stressed  an  operator,  or  lack  of  operator 
knowledge  cause  the  HPF  to  be  inaccurately  performed? 

4.9.2  Was  HPF  inaccuracy  due  to  the  operator's  misperception 
of  either  the  environment  or  the  components  of  the 
system? 

4.9.3  Did  HPF  inaccuracy  originate  in  operator  cognitive 
processes? 

4.9.4  Did  HPF  inaccuracy  originate  in  the  operator's 
ability  to  apply  his  motor  skills  to  the  system, 
as  designed?  That  is,  was  the  operator  unable  to 
manipulate  the  system  to  accurately  perform  the  HPF? 

To  maximize  measure  reliability,  determine  both  the  number  of 

subjects  to  be  run  and  the  number  of  trials  per  subject. 


TEST  CONDITIONS 


This  section  will  help  you  to  select  the  conditions  unddr  which  you  should 
measure  performance.  Notice  that  the  conditions  can  apply  to  the  System 
Performance  Issues,  as  well  as  to  the  test  measures  themselves. 

NEXT:  Turn  to  Human  Resources  Implications  and  Measures  (page  m-cif). 


CONDITION  CHART 


The  conditions,  in  the  following  ten  condition  classes;  are  those  which 
might  negatively  impact  the  operation  of  the  system  to  be  tested.  It  is 
assumed  that  best  case  conditions  will  be  tested  in  the  OT.  To  insure 
that  probable  and  operationally  significant  worst  case  conditions  are 
tested,  apply  the  Condition  Ranking  Scales  to  the  appropriate  conditions 
on  this  Chart. 


CONDITION  CLASS 

CONDITION 

Weather 

Clouds/Low  Cei 1 ing 

Rain 

Fog 

Snow 

Sleet 

Head  Wind 

Cross  Wind 

Tail  Wind 

Continuous  High  Wind 

Gusts 

High  Temperature/Low  Humidity 

High  Temperature/High  Humidity 

Low  Temperature/Low  Humidity 

Low  Temperature/High  Humidity 

Light 

Ambient  Light/Dark  Variation  (Natural  Light) 

Flares  and  Other  Artificial  Light 

Direct  Glare  (Natural  Light) 

Direct  Glare  (Artificial  Light) 

Reflected  Glare  from: 

Water 

Ice 

Snow 

Sand 

Clouds/Fog 

Ground  Surface 

Sandy 

Rocky 

Vegetation  Covered 

Muddy 

Loam  Soil 

Steppe 

Frozen 

Ice  Covered 

Snow  Covered 

Water  Covered 

Broken  Ground 

Plowed  Fields 

CONQITION  CLASS 


CONDITION 


Ground  Slope 


Obstacles /Concealment 


Target 


Operational 


Personnel 


Flat 

Hilly  (Positive/Negative  Slopes) 
Mountains  (Positive/Negative  Slopes) 


Dense  Forest/Jurigle 

Hedgerows 

Dense  Brush 

Sparse  Vegetation 

Water  Obstacles/Concealment 

Natural  Rock 

Holes/Di tches/Trenches 

Dense  Buildings/Walls 

Bridges 

Tunnels 

Manmade  Obstacles 


Range 

Size 

Figure-Ground  Contrast/Color 

Sound 

Speed 

Direction 

Altitude 

Maneuver 

Nature  of  Required  Hit  (Placement/Type/Number) 


Smoke 
Concealed 
Buttoned  Up 
100%  Loaded 
Damaged 

Reduced  Personnel 


With  low  but  acceptable  aptitude 
With  low  but  acceptable  demonstrated  ability 
With  minimum  acceptable  characteristics 
of  size  and  strength 
Extended  periods  of  work  without  rest 
(continuous  combatfatiqued) 

Wearing  clothing  and/or  gear  (e.g.,  winter 
gear,  NBC  protective  equipment) 


CONDITION  CLASS 


CONDITION 


Training 


Tactical : 


Following  a  period  of  time  without  training 
or  practice 
Trained  according  to: 

School  Training 

OUT 

Unit  Training 

With  little  or  not  field  experience  with 
this  system 

With  little  or  no  field  experience  with 
any  system  in  this  generic  class 
Crew  member  not  specifically  trained  for 
that  job 


Army  systems  are  exercised  according  to  the  tactical  theories 
prevalent  at  a  given  time.  Tactics  change,  but  the  parameters 
which  they  control  remain  relatively  constant.  The  tactical 
conditions  to  be  exercised  in  OT  will  either  be  the  complete 
tactic  or  that  sement  of  the  tactic  which  is  most  significant  for 
the  system  under  investigation.  The  following  parameters  of 
tactics  should  be  treated  in  the  same  manner  as  conditions  and  be 
subjected  to  the  Condition  Ranking  Scales.  The  resultant  ranking 
should  aid  in  determining  which  tactics  and  tactic  segments  to 
deal  with  in  OT. 


Movement  in  Attack 
Movement  in  Defense 
Movement  in  Travel 

Utilization  of  Terrain/Weather/Light 

Target/Target  Part  Selection 

Weapon/Ammunition  Selection 

Range  Selection 

Time  to  Fire 

Unit  Size 

Unit  Composition 

Unit(s)  Formation 

Unit  Coordination 

Inter-unit  Coordination 

Length  of  Continuous  Combat 


CONDITION  CRITICALITY  RANKING  SCALES 


To  rank  the  relative cri tical  i ty  of  conditions  within  each  condition  class, 
each  should  be  rated  according  to  the  following  parameters.  Assign  the 
numbers  zero  to  six  to  each  parameter,  as  indicated. 


What  is  the  probability  that 
the  system  will  operate  in 
this  condition  for  some  period 
of  time  during  the  sort  of 
operation  in  which  it  is 
expected  to  take  part? 


To  what  extent  was  this  system, 
or  a  major  component  of  this 
system,  specifically  designed 
to  operate  in  this  condition? 


To  what  extent  does  this 
condition  have  a  history  of 
being  associated  with  in¬ 
adequate  performance  and 
other  problems  in  similar 
systems  of  this  generic 
class? 


0  =  Probability  of  the  system 
operating  in  this  condition 
approaches  zero 

1 

2 

3  =  Probability  of  some  operating 

in  this  condition  is  about 
50% 

4 

5 

6  =  Probability  of  the  system 

operating  in  this  condition 
at  some  time  approaches  100% 


0  3  System/components  not  designed 
for  this  condition 

1 

2 

3  =  Design  takes  this  condition 

into  account 

4 

5 

6  =  Reason  for  the  design  of  this 

system/component  is  the 
ability  to  operate  in  this 
specific  condition 


0  *  No  history  of  problems 
1 
2 

3  =  History  of  a  number  of 

nonsevere  problems 

4 

5 

6  =  History  of  severe  problems 

and/or  frequent  problems 


To  what  extant  does  this 
condition  have  a  history  of 
being  associated  with  in¬ 
adequate  performance  and  other 
problems  in  the  system  in 
question  during  prior  OT's 
and/or  DT1 s? 


To  what  extent  will  this 
condition  place  unusual 
demand  or  stress  on  the 
operator  and/or  system? 


0  =  No  history  of  problems 
1 
2 

3  =  History  of  a  number  of  non- 

severe  problems 

4 

5 

6  *  History  of  severe  problems 

and/or  frequent  problems 


0  =  No  unusual  demands/stress 
1 
2 

3  =  Average  amount  of  demands/stress 

4 

5 

6  =  High  level  of  demands/stress 


CONDITION  DIFFICULTY  RANKING  SCALES 


To  determine  the  difficulty  of  performing  an  HPF  under  a  specific  condition, 
it  is  necessary  to  evaluate  the  difficulties  of  the  condition  present  in  the 
input,  decision  making  and  output  components  of  the  HPF.  These  component 
difficulties  sum  to  an  overall  difficulty  evaluation  for  that  condition. 
Assign  the  numbers  zero  to  six  to  each  of  the  following  parameters,  as 
indicated. 


1.  To  what  extent  does  this  condition 
increase  the  difficulty  of 
perception  of  the  external -system 
environment  for  this  HPF? 


Presents  no  additional  difficulty 


Somewhat  more  difficult 


Very  much  more  difficult 


2. 


To  what  extent  does  this  condition  0  = 

increase  the  difficulty  of  the  1 

internal  system  environment,  2 

including  its  displays  and  controls,  3  = 

for  this  HPF?  4 

5 


6  = 


Presents  no  additional  difficulty 
Somewhat  more  difficult 
Very  much  more  difficult 


3.  To  what  extent  does  this  condition  0  =  Presents  no  additional  difficulty 
increase  the  difficulty  of  decision  1 
making  for  this  HPF?  2 

3  =  Somewhat  more  difficult 

4 

5 

6  *  Very  much  more  difficult 


4.  To  what  extent  does  this  condition  0  =  Presents  no  additional  difficulty 
increase  the  difficulty  of  control  1 
of  motor  responses  for  this  HPF  2 

3  =  Somewhat  more  difficult 

4 

5 

6  =  Very  much  more  difficult 


To  what  extent  does  this  condition 
increase  the  difficulty  of  multiple 
demands  in  this  HPF?  (Multiple 
demands,  in  this  context,  relate 
to  time  sharing  of  perception, 
decision  making  and  motor 
responding. ) 


=  Presents  no  additional  difficulty 
=  Somewhat  more  difficult 
=  Very  much  more  difficult 
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HUMAN  RESOURCES  IMPLICATIONS  AND  MEASURES 


Human  Resource  Implications  are  those  questions  which  yoU  ask  to  aid  in 
determining  the  causes(s)  of  operator/maintainer  sub-cri terion  performance. 
Human  Resource  Measures  are  a  means  of  answering  the  Human  Resource 
Implications.  The  measures  of  Human  Resource  Implications  can  be  taken 
before,  during,  or  after  the  OT  exercise.  Whife''  the  OT  is  in  its  planning 
stage,  you  should  determine  when  each  measure  should  be  taken.  This  section 
includes  a  procedure  designed  to  aid  you  in  making  this  determination. 

During  OT  planning,  first  read  the  allocation  procedure  on  page  x-a.**  .  Next,  . 
read  the  Human  Resources  Implications  and  associated  Measures  on  pages  T 
and  apply  the  allocation  procedure  to  them.  Include  these  allocated  Measures 
in  your  Test  Plan.  If  you  discover  that  you  have  left  out  important  Human 
Resource  Measures,  immediately  following  the  OT  exercise  read  through  these 
Measures  again  and  select  appropriate  ones  which  you  can  still  take. 


1-28 


PROCEDURE  FOR  DETERMINING 
WHICH  HUMAN  RESOURCE  ISSUES 
TO  TEST  BEFORE  OR  DURING  OT 


STEP  1 


STEP  2 


STEP  3 


STEP  4 


STEP  5 


STEP  6 


STEP  7 


List  all  the  human  performance  functions  (HPF)  you  are  going  to 
test  in  OT  on  the  rows  of  the  Human  Resource  Issue  Work  Sheet 
Table  1  (pagers).  Make  additional  copies  of  the  worksheet  if 
necessary. 

Rate  each  HPF  on  each  of  the  four  rating  items  listed  on  page 
Place  the  items'  rating  in  the  appropriate  place  of  Table  1 
(page  a-#). 

Sum  the  ratings  for  the  four  items  for  each  HPF  and  enter  the 
value  in  the  appropriate  space  in  Table  1. 

Rate  each  human  resource  area  listed  in  Table  2  (page  Tt>t)  on 
each  of  the  three  rating  items  shown  on  page 

Sum  the  ratings  for  the  three  items  for  each  human  resource  area  and 
enter  the  value  in  the  appropriate  space  in  Table  2. 

Transfer  the  sum  values  from  Tables  1  and  2  to  the  margins  of 
Table  3  (page  x'}t)  • 

Multiple  the  marginal  values  in  Table  3  to  obtain  a  product  for 
each  cell  in  the  task. 


STEP  8:  The  cells  with  the  highest  values  should  be  considered  for 
measurement  before  or  during  OT. 


TABLE  2 


Human  Resource  Items 


Traini ng 


Human  Engineering 


Manpower  Planning 


Logistics 


Items  from  Page 


1 

1 

1 

2 

3 

Sum  1  to  3 

RATING  ITEMS  FOR  HUMAN  PERFORMANCE  FUNCTIONS 


To  what  extent  does  the  system  involve  a  modification  of  improvements 
which  could  impact  this  HPF. 

To  what  extent  have  similar  existing  systems, or  prior  OT  or  DT  activities 
demonstrated  problems  with  this  HPF. 

To  what  extent  is  it  anticipated  that  this  HPF  will  not  be  successfully 
performed  during  OT. 

To  what  extent  will  performance  of  this  HPF  be  evaluated  during  QT. 


RATING  ITEMS  FOR  HUMAN  RESOURCE  AREAS 


To  what  extsnt  have  similar  existing  systems  or  prior  OT  or  DT 
activities  demonstrated  problems  wiih  this  human  resource  area. 

To  what  extent  are  the  human  resource  measures  associated  with  this 
area  obtainable  after  the  OT  .xercise  has  been  completed. 

What  is  the  cost  and/or  feasibility  of  obtaining  the  measures  associated 
with  this  human  resource  area  before  or  during  OT  relative  to  the 
cost/feasibility  of  obtaining  them  after  OT. 


HUMAN  RESOURCE  ISSUES 
-  TRAINING  - 


The  following  measures  apply  to  those  Human  Performance  Functions  (HPF) 
for  which  performance  has  been  sub-cri terion. 

FOR  THE  HPF  WHICH  IS  AT  A  SUB-CRITERION  PERFORMANCE  LEVEL  APPLY  THE 
FOLLOWING  ISSUES: 

1.  To  what  extent  has  adequate  time  been  spent  in  training  on  this  HPF? 

1 . 1  Relationship  between  relative  HPF  criticality-difficulty  and 
relative  percentage  of  time  spent  in  training.  (See  procedure 
for  Diagnostic  Measure  #1,  page  1-4'.) 

The  relative  amount  of  time  spent  in  training  should  match  the 
relative  critical ity-difficulty  of  the  HPF.  This  measure  is  an 
attempt  to  determine  if  an  adequate  amount  of  training  time  was 
spent  on  the  HPF  in  question.  If,  in  relation  to  other  HPF’s, 
less  time  was  spent  training  an  HPF  than  its  criticality-difficulty 
indicates,  and  if  that  HPF  did  not  reach  criterion  performance 
level,  then  one  may  infer  that  insufficient  time  was  spent  on  it. 

2.  How  well  did  students  perform  this  HPF  in  training? 

2. 1  Mean  exit  score 


Exit  scores  are  those  numerical  ratings  which  are  assigned  to 
each  student  for  a  given  HPF  as  a  result  of  OT  training.  OT 
students  should  have  been  trained  on  selected,  critical  HPF' s  to 
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predetermined  levels  of  performance.  They  should  have  been 
given  tests  which,  when  performed  at  or  above  the  predetermined 
performance  1  evel s ,  permi tted  exi t  from  that  particular  part 
of  the  training.  Exit  scores  must  be  kept  as  raw  scores,  that 
is,  not  converted  into  pass-fail.  All  exit  scores  must  be 
recorded  for  this  measure,  regardless  of  the  students'  passing 
or  failing.  If  the  students  do  better  than  the  predetermined 
exit  criteria,  this  must  be  noted  and  not  collapsed  into  a 
homogeneous  pass  score.  Exit  scores  should  equal  or  exceed  OT 
HPF  criteria  for  successful  field  performance. 


2.2  Trainers'  ratings  of  student  ability 

It  is  assumed  that  student  ability  has  a  significant  impact  on 
training  effectiveness  in  OT.  It  is  also  assumed  that  exit 
scores,  though  important,  do  not  also  permit  sufficient  latitude 
for  an  understanding  of  the  students'  level  of  ability  in  OT. 

For  this  reason,  a  form  is  provided  to  aid  trainers  in  the 
assessment  of  student  ability.  Since  the  OT  training  package 
trains  personnel  who  come  into  the  OT- with  individual  ability 
levels,  an  .understanding  of  training  effects  necessitates  an 
understanding  of  pre  and  post  training  student  ability.  The 
appropriate  form  may  be  found  on  page 

Mean  number  of  exit  trials  to  reach  exit  criterion  in  training 

It  is  assumed  that  all  students,  in  the  OT  training  package,  may 
not  be  able  to  reach  HPF  performance  exit  level  on  their  first 
attempt.  The  mean  number  of  exit  trials  required  to  reach  exit 
criterion  provides  further  information  about  both  the 
effectiveness  of  the  training  and  the  ability  level  of  the 
students. 


2.4  Number  of  individuals  who  failed  to  reach  exit  level  criterion 


It  is  possible  that,  within  the  specified  number  of  exit  trials 
permitted,  some  students  may  not  be  able  to  reach  the  required 
level.  The  number  and  consequent  percentage  of  such  students 
will  provide  further  data  to  aid  in  the  determination  of  the 
effects  of  training  on  OT  performance.  If  a  large  number  of 
students  fail,  it  may  indicate  a  potential  difficulty  with 
the  HPF. 

To  what  extent  were  applicable  training  methods  used  to  train  this  HPF? 

3. 1  Compatibility  of  training  method  to  skill  being  trained 

The  skills  required  for  the  performance  of  HPF 1 s  may  be  trained 
by  a  number  of  different  methods.  The  nature  of  the  skills  ought 
to  determine  which  of  the  training  methods  is  used.  Sub-cri terion 
performance  in  the  OT  may  be  related  to  inappropriate  training 
method  used.  The  compatibility  between  training  method  and  skill 
may  be  viewed  as  affecting  OT  performance  in  the  following  manner. 
The  less  compatible  these  are,  the  greater  the  likelihood  of 
ineffective  learning  and  consequent  inadequate  performance.  For 
the  procedure  to  determine  this  compatibility,  see  pages 

How  effective  were  the  trainers  in  OT  training? 

4. 1  Applicable  training  experience  of  trainers  in  months 

There  are  certain  assumptions  one  makes  when  assessing  the 
effectiveness  of  trainers.  One  of  these  assumptions  is  that 
more  training  experience  should  increase  the  effectiveness 
of  training.  Knowing  the  amount  of  applicable  training 


experience  that  a  trainer  has  had  may  serve  as  an  aid  in 
explaining  student  OT  performance. 

4.2  Operator/maintainer  experience  or  training  on  current  or  similar 
systems  (in  hours  or  months,  as  applicable) 

It  is  recognized  that  individuals  who  are  not  highly  proficient 
at  a  particular  skill  may  still  be  able  to  train  others 
effectively  on  that  skill.  In  general,  however,  it  is  assumed 
that  there  is  some  positive  relationship  between  trainer 
operational  ability  and  ability  to  train  a  skill  effectively. 

4. 3  MOS  skill  level  of  trainer  in  applicable  specialty 

This  measure  may  be  viewed  as  a  subset  of.  4.2.  It  is  taken  for 
much  the  same  reason. 

4.4  Student  ratings  of  trainers'  abilities 

Student  ratings  of  trainers  serve  two  possible  functions: 

(1)  They  can  provide  information  about  the  trainers'  abilities, 
which  leads  to  a  greater  understanding  of  the  effects  of 
training  on  OT. 

(2)  They  can  serve  as  an  index  of  the  students'  attitudes  toward 
their  trainers,  and  this  itself  may  have  a  significant  bearing 
on  the  effect  of  the  training  package.  A  student  rating  form 
may  be  found  on  page 

5.  What  is  the  relationship  between  conditions  applied  in  OT  training  and 
OT  exercises? 

5. i  Similarity  of  conditions  applied  in  training  to  conditions 
encountered  in  OT  exercises 
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One  of  the  goals  of  training  is  to  prepare  students  for  the 
conditions  they  are  likely  to  encounter  in  the  operational 
environment.  If  students,  who  otherwise  pass  satisfactorily 
through  an  OT  training  course,  find  it  impossible  to  reach 
criterion  levels  in  the  field  exercises  of  the  OT,  one  may 
discover  that  they  could  not  make  the  transition  from  training 
conditions  to  field  conditions.  For  this  reason,  it  is  desirable 
to  determine  the  similarity  between  those  conditions  which  were 
applied  in  OT  training  and  the  conditions  actually  seen  in  the 
OT  exercises.  The  greater  the  disimilarity  between  those  two 
sets  of  conditions,  the  more  likely  training  is  to  be  a  cause  of 
sub-criterion  performance  in  OT.  Procedures  for  obtaining  this 
measure  may  be  found  on  page  x-rn. 

How  effective  was  the  organization  and  learning  environment  in  OT 

training? 

6. 1  Student  evaluation  of  training  experience 

As  in  4.4,  student  ratings  of  the  training  experienc  serve  two 
possible  funtions: 

(1)  They  can  provide  information  about  the  training  package,  in 
general,  which  may  not  be  available  through  more  standardized 
quantifiable  means.  This  information  will  tend  to  lead  to  a 
greater  understanding  of  the  effects  of  training  on  OT. 

(2)  They  can  serve  as  an  indication  of  students'  attitudes  toward 
the  OT  training.  These  attitudes,  whether  valid  or  not,  will 
influence  the  effect  of  the  OT  training  package  on  the  students 
and,  therefore,  provide  further  information  on  the  reasons  for 
sub-criterion  OT  performance.  A  student  rating  form  can  be 
found  on  page 


5.2  Trainers'  evaluation  of  the  OT  training  experience 

(1)  They  can  provide  information  about  the  training  package,  in 
general,  which  may  not  be  available  through  more  standardization 
quantifiable  means.  This  information  will  tend  to  lead  to  a 
greater  understanding  of  the  effects  of  training  on  OT. 

(2)  They  can  serve  as  an  indication  of  trainers'  attitudes  toward 
the  OT  training.  These  attitudes,  whether  valid  or  not,  will 
influence  the  effect  of  the  OT  training  package  on  the  trainers 
and,  therefore,  provide  further  information  on  the  reasons  for 
sub-criterion  OT  performance.  A  trainer  rating  form  can  be 
found  on  page 
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PROCEDURE  FOR  DIAGNOSTIC  MEASURE  #1 
(Relationship  between  relative  HPF  criticality-difficulty  and 
relative  percent  time  spent  in  training). 

Step  1.  Determine  the  criticality  for  each  HPF  involved  in  OT  training 
(regardless  of  whether  performance  in  OT  was  sub-criterion) . 

1.1  Rate  each  HPF  on  each  item  of  the  criticality  rating  form 
contained  on  page 

1.2  Write  the  individual  criticality  ratings  on  the  worksheet 
contained  on  page  .  (Photocopy  a  sufficient  number  of 
worksheets  to  accommodate  all  HPF's  to  be  evaluated). 

1.3  Total  the  ratings  for  each  HPF  and  enter  the  values  in  the 
appropriate  spaces  on  the  worksheet. 

Step  2.  Determine  the  difficulty  for  each  HPF  involved  in  OT  training 
(regardless  of  whether  performance  in  OT  was  sub-criterion). 

2.1  Rate  each  HPF  on  each  item  of  the  difficulty  rating  form 
contained  on  page 

2.2  Write  the  individual  difficulty  ratings  on  the  worksheet 
contained  on  page  t-h'1 

2.3  Total  the  ratings  for  each  HPF  and  enter  the  values  in  the 
appropriate  spaces  on  the  worksheet. 

Step  3.  For  each  HPF,  multiply  the  criticality  sum  by  the  difficulty  sum 
to  obtain  the  product  of  sums;  then  enter  the  values  in  the 
appropriate  spaces  on  the  worksheets. 
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Step  4.  Rank  tne  product  of  sums,  using  "1"  for  the  highest  product; 
this  is  the  most  critical/difficult  HPF. 

Step  5.  Obtain  from  the  OT  training  personnel  the  mean  amount  of  training 
time  spent  on  each  HPF. 

Step  6.  Enter  the  training  time  for  each  HPF  in  tne  appropriate  place  on 
the  worksheet. 

Step  7.  Rank  the  training  times,  using  "1"  for  the  most  time. 

Step  8.  Compare  the  ranked  product  of  sums  with  the  ranked  training  time. 

8.1  If  the  ranked  product  of  sums  is  larger  or  equal  to  the 
ranked  training  time,  then  one  may  conclude  that  adequate 
training  time  was  given  on  this  HPF. 

8.2  If  the  ranked  product  of  sums  is  smaller  than  the  ranked 
training  time,  then  one  may  conclude  that  insufficient 
training  time  may  have  been  given  on  this  HPF. 


HPF  CRITICALITY  RATING  FORM 


To  compute  the  criticality  of  HPF's  for  training,  each  should  be  rated 
according  to  the  following  parameters.  Assign  the  numbers  zero  to  six 
to  each  of  the  following  parameters,  as  indicated. 


1.  To  what  extent  is  this  system, 
or  a  component  of  this  system, 
specifically  designed  to  carry 
out  the  HPF. 


2.  In  relation  to  other  HPF’s  how 
often  is  it  performed  in  this 
system? 


3.  What  would  be  the  effect  of  failure 
to  successfully  complete  this  HPF 
on  survivability  of  the-  system/ 
operator? 


0  =  Not  designed  specifically  for 
HPF 

1 
2 

3  =  Design  takes  the  HPF  into 

account  but  is  not  specifically 
designed  for  it 

4 

5 

6  =  Principle  reason  for  the  design 

of  this  system  is  the  performance 
of  this  specific  HPF. 


0  =  Seldom  if  ever 
1 
2 

3  =  About  as  often  as  any  other 

4 

5 

6  =  This  system  will  only  perform 

this  HPF. 


0  =  No  effect 
1 
2 

3  =  Medium  probability  of  destruction 

4 

5 

6  =  High  probability  of  destruction 

or  surrender  of  system 


To  what  extent  have  other  systems 
in  this  generic  class  had  a  history 
of  inadequate  performance  or 
encountered  problems  in  carrying 
out  this  HPF. 


To  what  extent  have  prior  DT's/ 
OT's  indicated  problems  in 
successfully  performing  this 
HPF. 


What  would  be  the  effect  of 
failure  to  perform  this  HPF 
on  the  effectiveness  of  the 
system's  mission? 


0  =  No  history  of  problems 
1 
2 

3  =  Has  encountered  a  number  of 

severe  problems 

4 

5 

6  =  History  of  severe  problems 

and/or  frequent  problems 


0  =  No  history  of  problems 
1 
2 

3  =  Has  encountered  a  number  of 

severe  problems 

4 

5 

6  =  History  of  severe  problems 

and/or  frequent  problems 


0  =  No  effect 
1 
2 

3  =  Mission  significantly  impaired 

but  not  a  complete  failure 

4 

5 

6  =  High  probability  of  total 

mission  failure 


HPF  DIFFICULTY  RATING  FORM 


To  determine  the  difficulty  of  an  HPF,  it  is  necessary  to  evaluate 
difficulties  of  its  input,  decision  making  and  output  components, 
component  difficulties  then  sum  to  an  overall  difficulty  evaluation 
Assign  the  numbers  zero  to  six  to  each  of  the  following  parameters, 
indicated. 


To  what  extent  is  perception  of 
the  external -system  environment 
a  source  of  difficulty  to  this  HPF? 
Factors  contributing  to  this 
difficulty  include: 

0  ~  Presents 

1 

2 

3  =  Somewhat 

4 

C 

no  difficulty 

difficult 

(a) 

vibration 

3 

6  =  Very  difficult 

(b) 

inadequate  illumination 

(c) 

inadequate  visual  access 

(d) 

too  much  noise 

(e) 

many  inputs  to  attend  to 

- 

To  what  extent  is  perception 
of  the  internal  system  environment, 
including  its  displays  and  controls, 
a  source  of  difficulty  in  this  HPF? 
Contributing  factors  include: 

0  =  Presents 

1 

2 

3  =  Somewhat 

4 

C 

no  difficulty 

di ff icul t 

(a) 

vi brati on 

6  =  Very  difficult 

(b) 

inadequate  illumination 

(c) 

poorly  designed  on  placed 
displays/controls 

(d) 

inadequate  visual  access 

(e) 

many  inputs  to  attend  to 

(f) 

too  much  noise 

3. 


To  what  extent  is  decision  making  0  =  Presents  no  difficulty 
a  source  of  difficulty  in  this  1 


HPF? 

Contributing  factors  include: 

2 

3  =  Somewhat  difficult 

(a) 

large  number  of  decision 

4 

points 

5 

6  =  Very  difficult 

(b) 

many  alternatives  per 
decision 

(c) 

lack  of  formal  decision 
rules 

(d) 

tight  time  constraints 

(e) 

important  consequences 

4.  To  what  extent  is  the  control  of  0 
motor  responses  a  source  of  1 

difficulty  in  this  HPF?  2 


Contributing  factors  include: 

3  = 

A 

(a) 

vibration 

4 

5 

6  = 

(b) 

improper  design  of  a  control 

(c) 

poor  position  of  control 

(d) 

short  time  constraints 

(e) 

close  tolerences  required 

(f) 

large  amount  of  strength  required 

(g) 

many  responses  required 

Presents  no  difficulty 
Somewhat  difficult 
Very  difficult 


5.  To  what  extent  are  multiple  demands 
inherent  in  this  HPF  a  source  of 
difficulty?  Multiple  demands  in 
this  context  relate  to  time  sharing 
of  perception,  decision  making  and 
motor  responding. 


0 

1 

2 

3 

4 

5 

6 


=  Presents  no  difficulty 


=  Somewhat  difficult 


=  Very  di ff icul t 


WORKSHEE 


DIAGNOSTIC  TRAINING  MEASURE  #3 
COMPATIBILITY  OF  SKILL  AMD  TRAINING  METHOD 


Step  1 


Step  2 


Skill 


Skill 


Enter  percent  of' time  spent  training  this’HPF  in  each 
modality  listed  below: 

Modal ity  %  Time  Spent  in  Training 

1.  Classroom-lecture  — — - 

2.  Audio-visual  presentation  - 

3.  Paper  Pencil  Test  — - - - 

4.  Hands-on  simulation  — - 

5.  Field  Experience  •  - 

6.  Observation  of  other 

students.  - 

.  Compute  a  time  index  for  each  of  the  _ _  skills  listed  below 

using  the  equations  given. 

where  =  %  of  time  spent  in  training 
this  HPF  using  modality  1  (cl assroom-1 ecture) 

~  %  of  time  spent  in  training 
this  HPF  using  modality  2  (audio-visual  presentation) 
and  so-on  for  each  modality  listed  in  Step  2. 

1  (Visual  Discrimination)  Index  = 

(m,  x  1.1)  +  (55M2  x  5.8)  +  (%M3  x  4.0) 

+  (2M4  x  3.0)  +  (%M5  x  2.8)  .... 

2  (Fine  hand  motor  control)  Index  = 

(%M1  x  .1)  +  (SM2  x  .2)  +  (o;M3  x  .1) 

+  (%M4  x  5.4)  +  (%Mg  x  6.8)  +  .  . -.  . 


T  -  4  R 


Skill  3  (Team  Coordinator)  Index  = 


Skill  N  ( 


)  Index  = 


Step  3.  Enter  the  percent  of  HPF  performance  that  involves  each  of  the 
skills  1 isted  below. 


Skill 


%  of  HPF  Performance 
Involving  the  Skill 


1.  Visual 

2.  Fine  hand  motor  control 

3.  Team  Coordinator 


N 

Step  4.  Rank  the  skill  time  indices  (computer  in  Step  2)  from  lowest 
(Rank  =  1)  to  highest  (rank.=  N)  and  enter  them  on  the  chart 
below. 

Rank  the  %  of  HPF  performance  involving  each  skill  (computed 
in  Step  3)  from  lowest  (rank  -  1 )  to  highest  (rank  =  N)  and 
enter  on  the  chart  below. 

Skill  Time  %  HPF  Performance 
Skill  Index  Rank  _ Rank _ 


1.  Visual  Discrimination 

2.  Fine  hand  motor  control 

3.  Team  Coordinator 


rdance  between  the  two  sets  of  ranks 
value,  the  better  the  compatibility 
ed  to  train  this  HPF. 


TRAINING  DIAGNOSTIC  MEASURE  =3 
CONDITIONS  IN  OT  TRAINING  AND  OT  EXERCISE 

Step  i.  List  the  condi tion(s)  under  wnich  performance  on  this  HPF  failed 
in  the  operational  test 

Condition  in  OT 

1.  - 

2.  - 

3.  - 

4.  - 

Step  2.  Determine  the  percentage  of  time  this  HPF  was  practiced  in  each 
condition  identified  in  Step  1. 

Condition  %  Time  Practiced 

(from  Step  1 )  in  Training 

1 . 

2.  -  - 

3.  -  - 

4.  -  - 

Step  3.  Was  there  any  condition  under  which  no  practice  on  this  HPF 

took  place  in  training  (that  is,  are  there  any  zero  percentages 
listed  in  Step  2)? 

If  the  answer  is  _  then: 

Yes  may  indicate  that  HPF  performance  was 

suboptimal  because  no  practice  was  given 
under  this  condition  in  OT  training. 
Continue  with  Step  4. 


No 


"1 


Step  4.  Compute  the  criticality  of  each  condition  using  the  HPF 
condition  criticality  rating  form  on  page  .  Enter  th< 
below. 


Enter  the  values 


Condi tion 
from  Steo  1 


Rating  Form 
Rating  Form  Items 


1  to  5 


Step  5.  Compute  the  difficulty  of  each  condition  using  the  HPF  condition 
difficult  rating  form  on  page  .  Enter  the  values  below. 


Condition 
'rom  Step  1 


Rating  Form 
Rating  Form  Items 


1  to  5 


Step  6.  Compute  for  each  condition,  a  single  criticality-difficult  index 
by  multiplying  the  criticality  and  difficulty  ratings  from  Steps 
4  and  5.  Rank  these  giving  a  rank  of  1  to  the  lowest  product. 


Condition 
from  Sceo  1 


Step  4 
Sum  1-6 


Step  5 
Sum  1-5 


=  Product 


1-52 


Step  7.  Rank  the  percentage  time  values  in  Step  2  and  compete  the  design 
of  concordance  between  tress  ranks  anc  _hL.ce  csrncs^ed  in  Stes 


Condi ticn 


Percentage 
Time  Ranks 


Ranks 

From  Sten  6 


1-53 


